Hello everyone, thank you for joining. My name is Pierre Rathmerck, I'm a PhD candidate
at Fusek at Friday, University of Amsterdam, and I'm in the Security Research and mainly
specializing in UEFI PCI Express and Hypervisor Security. My free-fuse work involves
intultenable vulnerability research, and this is my colleague Sina.
Hello everyone, I'm Sina, I'm also a PhD candidate at Friday University at Amsterdam.
I usually spend my time on system programming, maintaining HyperdvG and like Windows
internals and a couple of digital designs, you can see more of my works here at my
blog. So we're giving two talks today at Fusek. The current one is an introduction
to HyperdvG where we go into details on what it can offer you, but also how it works
on the do-hot. And this morning we gave a talk in the Security Track where we discussed
the case study of using HyperdvG for malware reversing. If you missed that one, no worries,
that will be recording. Yeah, so let's talk about HyperdvG. We are currently sitting in
the virtualization track, but we're talking about B-buggers, so what's up with that?
Well, actually we're going to talk about both HyperdvG and D-buggers. So why would you
want a HyperdvG based debugger? Well, basically a HyperdvG is highly preferred. It has
complete system visibility, and that means that it has visibility over nearly all events
that are occurring in the operating system. At the same time, because it is operating
at the HyperdvG level, it is also virtually completely transparent to both user and kernel space.
In this case, all sources of advantages, one being stealthy. This is, of course,
very useful when the debugging software that behaves differently when running in a virtualized
environment or when being debugged. And yeah, being a HyperdvG means we also have access to
a toolbox that offers features that are normally not possible with traditional debuggers.
So, introducing HyperdvG, it's a Hypervisor assisted debugger, leverages virtualization
extensions offered by the Intel ISA, originally intended for virtualization, but we're using them
to implement a debugger. And because we are operating at the Hypervisor level, we operate also
independently of any operating system level debugging APIs. This gives all sorts of
advantages and we'll be talking about those in a more detail. The first release was for Windows
back in 2022, actively maintained since, and we're currently working on a UEFI based,
always agnostic, hypervisor agent, which means that soon we'll be able to support Linux,
BSD, and basically any other operating system. So, yeah, let's talk about some debugging scenarios.
Like, if you have to debug native code, especially when you don't have access to the source code,
this can be really difficult. For example, debugging a device driver. You're debugging the
device driver, or you're actually looking for a device driver that is touching a certain
area in the memory, and you don't know which device driver it is. Well,
this is exactly one of the use cases that's hyperdvG is useful for. So, you try to find what
device driver, or what user space program is serving, writing into the certain memory range,
no problem, you can find that out. When you have finally found what device driver is writing into
that certain memory range, you might want to reverse trace the stack all the way up to user space.
You know, maybe there's a user space companion app talking to the device driver. This is also possible
with hyperdvG. Maybe you want to script all of this, right? Turn all of these
data points into events that triggers certain scripts, and does something interesting.
Or maybe you want to prevent user kernel space, touching a range of memory.
And what the list goes on and goes on, these are all basically use cases that are not
possible with a traditional but debugger, and we'll show you how this can be done with hyperdvG.
So, hyperdvG to the rescue. But before we do that, we have to introduce some new terms.
So, hyperdvG implements a concept, but we like to call event-driven debugging.
So, everything that happens in hyperdvG is basically an event, and each event could trigger
one or more actions. So, it could be the execution of some script, it could be the execution of
a piece of assembly code that you wrote beforehand, and you could simply trigger a breakpoint.
And the inputs for these events can be anything from system calls, or returns from system calls,
to EPT hooks, what that is, will dive into in a little bit. It can also be IO operations,
and it can be even be specific CPU instructions. Like you know, that the certain piece of software
either user or kernel code uses these instructions. You can actually trigger an event based
on the use of this only instruction. So, there are event calling stages, like post, pre, and both.
You can also do what we call like to call event short circuiting, which basically means you
define a set of conditions under which the event must be triggered, and then you can simply
ignore the entire event. This is how you can ignore read or write actions to a certain
memory region, for example. You can also completely bypass events, which means that
a certain range of instructions will be skipped. And we implement all of these using
PMXids, but we'll talk about that. Yeah, and so let's dive a little bit deeper into
how we implement it all of this.
No, we're going to talk about the techniques, the way that we implement these features in HyperDVG,
and how they can help us to enhance or reverse engineering experience.
The first feature that we offer here is by using monitor trap flag. If you are familiar with
virtualization of Intel VTX, there is like a VMCS and on top of that there is an MTF flag.
We made like a function tracking mechanism here that you can use to
go from the user mode to kernel mode and directly back from the kernel mode to user mode by
employing MTF. And once it's combined with a symbol server, let's say Microsoft symbol server or
a private symbol server, PDB files, you can just see the name of the functions, the call,
three of everything that is called, and HyperDVG is capable of running from user mode to kernel mode
and coming back from the kernel to the user. This is something that traditional
devices are not capable of doing that. So in another scenario, we are trying to combine
what we introduced before about even calling the stages and even short circuiting.
There are three scripts, there are three HyperDVG scripts. All you can see here is
from the language of HyperDVG, we call DSLank, it's a customized language which is so similar to
WindyVG scripting language, but with a little bit of difference. So this is the way that we create
an event in HyperDVG. As you can see, we are trying to bypass the execution of certain instructions.
So first, we go for MSORI, which is basically the WR MSORI, if you're familiar with the
model specific registers in Intel. And here we set like in the, if you see the stages here,
we defined two stages as a pre-estate and post-estate. And because the first example is a pre-estate,
so the WR MSORI instruction is not yet executed and as a result, we could change the values that
the WR MSORI instruction wants to put inside like wants to run on a bare metal machine. So they are
all executing before the emulation happens in the system. Another example is exceptions. You know,
there are plenty of like, you know, hypervisor, there are exceptions, there are interrupts, there are traps.
All of them can be monitored and can be emulated by HyperDVG. So in this example, we use a post-calling
stage here and as a result, like 0XE is a page fault. And the based on Intel processors,
there is like a location where the page fault happened in CR2, which is there. So because we are using
the post-estate, we could see where this page fault happens and like see the address.
Another example is about changing system calls. So we have two examples here. For the first
example is that we are trying to intercept the Cisco instruction. And for the Cisco instruction,
first we check whether the Cisco number, which is always located in both Linux and Windows,
all located on the first EA 3G server, we check it with some values. Let's say 0XE55.
And if it was like, if the Cisco number is 55, then we change one of the parameters. Like,
if the assuming that it's a Cisco, it's a fast-call calling convention, we change some bits
in ECX register or first parameter. And as you can see, there is also another example here
that we call it as the previous stage. So we have the capability to bypass the system call.
What happens here is that HyperDVG before executing and before emulating the Cisco instruction
checks for some conditions. And if this condition is like the condition, if the condition
met, then this event is short-circuited. Which means that the Cisco instruction is never
emulated and never executed in the virtualized environment. And what we do here, we change the
REX register to put some status here. So the application, the debug application, things that
like the Cisco instruction is successfully executed. And the value in REX, which probably
refers to like access denied file is not able to be accessed. It's returned from the system called
button reality. There is no Cisco and nothing is executed in the system. So this is how you can
use short-circuiting. And you know, like you can also combine short-circuiting with all of the events
in HyperDVG. Let's say for events that are related to WDRMSR, RDMSR, CPUID, TSC,
like RDSC and RDSCP or RDPMC, all of the events that are available in HyperDVG could be
short-circuited. And another interesting example here is that how we can ignore
certain memory rights. Here by implementing this, we use the EPD, second-level page table implementation
of Intel. And as you can see, we put like a hook inside a certain address. Let's say it's a
variable inside a user mode application or it could be also in a kernel mode like driver.
And we check for all of the modifications like any memory rights into this address or memory
reads into these address. So if there are, because it's W here, it means that we want to
intercept and trigger some events for memory rights. And we check for this stage. So if the memory
is equal to something like let's say a specific value, then we could easily change that value
as if nothing happens and like we completely change the value of the memory. How it could be useful
as soon that you have a program that has like different threads, different threads that are accessing
a specific global variable. And you just want to make sure that this specific global variable
is never changed to a certain value. Using this script, you can short-circuit or you can
see or modify or prevent any modification of the target memory. So what make these things possible here?
We extensively use the MVAC or mode-based execution controls, which are available on Intel
processors for VTX. We have a mechanism in which we like allocate and use three different
EPDPs. So different EPD pointers. And one of these EPDs are a normal EPD and the other EPDs
like a user mode execution denied EPD and the other one is like kernel mode execution denied EPD.
Regarding MVAC, MVAC is introduced in Intel processors starting from KBLAG processors. I think it's
seventh generation. And whenever we reach to our target process, we just like detect whether
we want one of these user denied or kernel denied EPDP's in VMCS. So what we have here is that
we combine or detect more changing detection, mechanism with some other features of Intel,
which is called move to CR3 exiting. The thing is, whenever the CR3 register is changed, we are
sure that some context switches happens in the process. And based on that, if we are interested
on that process, we will load one of these three EPDs. So if the process is interesting for us,
then we load one EPD that detects, for example, the execution of user mode.
Now, like using this approach, using MVAC makes us capable of freezing the execution
of applications. Assume that we are targeting a specific process and we are loading EPDPs for
that specific process that doesn't execute any user mode code. So what will happen here is that
the operating systems context switches to this specific process. And we only understand it by
intercepting all of the VMXs that are related to move to CR3 exiting. And in the context
switch, we just load the EPDP that denies the execution of user mode code. So what happens is that
at some point windows or the operating system tries to execute some user mode codes. And what will
happen is that a processor creates some EPDP violation of VMXs. And in the EPDP violation,
VMXs, it will just basically ignore it. As if we don't let the application to run. And at some
points, the clock interrupt comes and changes the process to another process. So what happens is that
what happens here is that the operating system thinks that the user mode code is running
normally. But in reality, we just prevent any execution of user mode codes. And this is the
way that we implement the time freeze debugging for the user mode applications in HyperDouge.
Here for the conclusion, HyperDouge has two different mechanisms both for debugging user mode and
kernel mode. So you can just debug an entire operating system with it. And it also leverages
modern hardware features that gives it a system wide visibility. So it powerful, it provides
some powerful features that are designed for reverse engineering. And these features are
simply not possible with regular debugger, traditional debugger. So you need a hypervisor to
have those reverse engineering techniques. And of course HyperDouge is a free and open source
project. So it's available for the community to contribute and has and patches are always welcome.
Yes, that's it.
This is the on the low level of instrumentation that you just do. Do you have any higher level
utilities to provide a bit more for example, OS awareness? Like I remember looking into
the open VMI previously. So you mean the difference is that HyperDouge and open VMI?
Yes, maybe you can put it in context. Yeah. Yes, the question is what higher level features we offer
for seeing some higher level operating system content? We do have like a
specific script engine that is written for HyperDouge. And this script engine has a lot of
like it adverse about some like operating system concepts like thread ID process ID and it also has
like it has access to call certain let's say functions from the Windows kernel or it
is it has completely a complete interaction with the operating system. So it's the higher level
of debugging. Yeah, I would like to add that you can also use the Microsoft Symbol server
or your own private symbols to give context to the coach you're looking at.
Yes, so we're working on a UEFI based agent. So basically that means
our hypervisor will run at boot time and that way we can support Linux and be as the
and all the other operating systems. But it's a work in progress. So yeah, if you like helping us out,
please do. And also maybe it's good to add that in HyperDouge like the way that we designed
this script engine is completely different from the Vindouge. And we published an academic paper
out of it and if you see that academic paper, it'll first like more than 1,000, it is more
than 1,000 times faster than Vindouge because of its design.
Another question? Yeah, to expand on that. So when you say this is working progress,
I like working to have second line, like kind of individual space,
second line of all these different operating systems.
So the question is, can you? Are you going to implement current and user space,
second line for all of the different operating systems you mentioned, like how are you going to
use those symbols and understand what is going to be the space? Yes, so the question is if you are
going to support and add the support for all of the operating systems. Am I right?
Yes, actually the thing is like there are certain things in HyperDouge that are
not related to the platform. And there are something that are related to the platform. For example,
we are using for now, we are using Intel VTX. So we have to execute those instructions that are
related to the VTX or VTD. But the thing is, like there are also different, like we have also
used some mechanisms that are only available on Windows. Let's say IRQLs. But the thing is,
all of them can be applied to other operating systems as well. Like for example, for IRQL,
there is nothing like this in Linux. But we are trying to implement those specific features for
other operating systems as well. So you have one example you have of like system call
interesting. So you have for instance something where I can say I want to figure out every open
system called a Linux. Like you have this information about what the system call is in Linux.
So the question is if we have like information about what the open system calls are in Linux,
right now. I mean, what what we did for implementing system calls can be applied also in Linux.
Like we are all of the events in hardware DVG are using hypervisor capabilities. So there is
no like it doesn't relate to Windows. But I mean, the user should also know about the way
that the system call is implemented in Linux. For example, I assume that Windows uses a specific
calling convention. Let's say fast call. And in Linux, the user should also know about the fast call
and the difference between fast calling and in Linux and Windows and trace those system calls.
But right now does hypervugiorate and the standard calling convention in Linux?
No, no, right, right now hypervugiorate only understands Windows.
So you mentioned your infinite time like the time we're doing or in this space process is only
how stealthy is that? Because as far as I understand like the time like you're not
turn on space, right? So if a turn on space, a malware would monitor CIS calls, would they be able to
catch it? So the question is how stealthy is the time freezing debugging inside like Windows?
Because we are only time freezing user mode applications. You want to know whether like a
kernel malware could bypass this or not? The thing is in hypervugiorate, there are two modes of debugging.
Whether you can debug the kernel entirely the kernel and pause everything like pause the operating system
and nothing changes in the operating system and the other thing is the user mode code is like the
user mode debugging is a separating and the thing is like if there is a malware that could run
into the kernel then there are like some limitations for those malware like for example in Windows
there is a driver signature enforcement so they have to sign it their driver and there are also
other ways of like for example who king those functions that are designed to load certain
their drivers so we can just catch them from there. Yes? Yeah so the question is we can hide
all of those footprints or not of the hypervisor. The thing is or first talk in this morning
in the security track was exactly about this thing. We cannot guarantee a 100% transparency
but we like raise the bar a lot like hypervugiorate even though by its nature is more steals
because it simply doesn't use API that Windows API that are designed for debugging but at the
same time we also try to mitigate those footprints that are for hypervugiorate itself.
So let me quickly pull up a slide from this morning that exactly answers your question
one second.
So this is the roadmap that we have for mitigating all of the artifacts that hypervisors
ourselves but also the top level hypervisor could reveal to malware. We have the two on the left
we have implemented and we're working on the ones in the middle there about 75% done and the
remaining ones yeah there's scheduled right there. Do we have time for more questions or yeah?
Sorry can you speak up?
Yes well we don't currently don't have plans for that but it is certainly possible to implement
to say on arm and aim D. Yes go ahead as a member you know we have some commands like
extension commands for SMIs and we kind of could like
monitor certain things for the SMI handlers for example if I don't know an attacker or any
other application or the Windows itself tries to call and tries to trigger some SMIs by using
IO port certainly you can use the features in hyperduty to detect that and we are also having
some commands for triggering SMIs like you just trigger the SMIs by using hyperduty if you
check the document there is the certain commands for SMI handlers but in general everything is
inside the VTX inside the ring minus two hypervisor not in SMM.
No no I mean this is not possible in interpasses or like you have to have like access to the
UEFI firmware that loads the SMI handler and your code should be in SMM so this is technically not possible.
This is actually part of our effort to port hyperduty to UEFI we are currently not sure yet
but it's feasible but since we will be running from UEFI we might be able to actually intercept
anything that's happening in SMM. But again like running into UEFI is like you need to have like
some access to the SPI chip because like the SMI handler is loaded before the UEFI application yes
is the QMU I mean what we could do for the QMU is that we could we do use some of their commands
like some of their codes for emulating certain instructions but here the like what we do is that
we execute like we run on bare metal so we don't have the plan to.
Yeah yeah I mean like right now hyperduty is supported on the VMware workestations nested virtualization
environment and KVM like nested virtualization environment it's not so it doesn't support hyperv like
we try a lot but unfortunately we couldn't port it to the hyperv yet but right now I think KVM
supported yeah so the overhead comparison normal hypervisors I mean this is like the way that
we emulated is exactly same as the way that you emulate like let's say a VMware like I mean once
we execute those instructions I think we didn't have any measurements for this but I think
it should be generally with less overhead compared to like big hypervisors like KVM because
or VMX it handler has a much shorter path and it's like the code that we wrote for it is of course
shorter than what they wrote for KVM but I mean in any case if you are running hyperduty in a nested
virtualization environment let's say on a VMware then the overhead of hyperduty is also added to the
overhead of the VMware workestation nested virtualization because Intel doesn't support officially doesn't
support nested virtualizations all they just emulate everything so there is no nested virtualizations
support