How to implement __try __except in ML64 (MASM)
I'm currently working on a project that is requiring me to write a bunch of x64 assembly code. One of the requirements that surfaced is that I needed to be able to trap exceptions in a specific block of code in the same way that __try / __except works in C++. As it turns out, figuring out how to do this is far from obvious.
If you aren't familiar with how x64 exception handling works, the msdn documentation can be found here: https://msdn.microsoft.com/en-us/library/1eyas8tf(VS.80).aspx. The Portable Executable / COFF specification can be found here: https://download.microsoft.com/download/e/b/a/eba1050f-a31d-436b-9281-92cdfeae4b45/pecoff.doc. In addition, here is an excellent blog entry that details the entire x64 Application Binary Interface: https://blogs.msdn.com/freik/archive/2006/03/06/X64_calling_conventions_summary.aspx. Finally, for some background on Win32 structured exception handling, here is a link to Matt Pietrek's article: https://www.microsoft.com/msj/0197/Exception/Exception.aspx .
So with that background, I quickly realized that MASM's PROC directive supports the FRAME parameter which allows you to specify an exception handler. Using FRAME will result in the correct metadata getting written to the RUNTIME_FUNCTION table in the executable. So I dutifully tried that and it worked like a champ. The only trouble was that function scope wasn't what I needed; I needed to trap exceptions at the line level.
After hours of reading every article, blog, and forum post on x64 exceptions on the internet, I still didn't have the answer. My best guess was that Visual C++ was using some sort of language specific handler to implement __try __except, but I could find no details other than the prototype for the handler (which looks like this):
typedef EXCEPTION_DISPOSITION (*PEXCEPTION_ROUTINE) (
IN PEXCEPTION_RECORD ExceptionRecord,
IN ULONG64 EstablisherFrame,
IN OUT PCONTEXT ContextRecord,
IN OUT PDISPATCHER_CONTEXT DispatcherContext
);
Then it finally dawned on me: if the language specific handler has a prototype, it has to be a function. Well, the ehandler for the FRAME parameter of PROC is also a function. Sure enough, when I looked at the contents of the registers when my handler was called, the contents matched the prototype for the language-specific handler.
With that, I realized that I had been framing the problem wrong. I had assumed that the answer lay in finding some way to apply the handler in a more limited scope. As it turns out the answer is to handle the scoping *within* the handler. The key to this lies in understanding how the handler itself works. Once execution is transfered to the handler, execution behavior is ultimately determined by the return value of the handler. The possible return values are described in the EXCEPTION_DISPOSITION enum, but the most important options are ExceptionContinueExecution and ExceptionContinueSearch. If the former is returned, context is restored to the point where the exception occured and execution continues. If the latter is returned, then the system continues searching for a handler for the exception.
Since the handler can decline to handle the exception, implementing __try __except becomes a relatively simple matter of determining whether the line of code that triggered the exception lies within the guarded section. If the exception occured from with the guard block, then it gets handled and execution can continue. Otherwise, the handler will decline to handle the exception and the search for a handler will continue. Determining the location where the exception occured is trivial since that information is stored in the EXCEPTION_RECORD that is passed (by reference) to the handler. In addition, (among other things) the handler gets passed a pointer to a CONTEXT. The CONTEXT has a complete description of the execution context at the time the exception occured. For example, all of the register states are stored. Modifying the context is one way that the cause of the exception can be addressed so that execution can continue.
The other interesting parameter is the DispatcherContext. While the EXCEPTION_RECORD provides the location where the exception actually occured, it is important to understand that the exception may not have been generated by any of the code in the frame associated with the handler. In the case of implementing a __try __except block, the information that we need to know is where control actually left our frame. This information is provided by the DISPATCHER_CONTEXT. The ControlPc member provides the location of the instruction pointer where control left the frame (or where the exception actually occurred if it occured in the frame).
Here is some code to go with the discussion. Below you will find a function called TestExceptionHandling along with an exception handler (ExceptionFilter). TestExceptionHandling simply causes an access violation to occur by dereferencing NULL. This is done within guard labels. When the exception filter is called, it checks to see if the exception was generated within the range identified by the guard labels. If so, it jumps to the exception handling code within TestExceptionHandling. This code changes the context so that the memory being dereferenced is valid and then continues execution.
In this implementation, the "__except" implementation is actually in TestException itself. I could have just as easily done this from within the handler, but I was trying to code something that followed the __try __except semantics. In addition, this code does not use the DISPATCHER_CONTEXT to locate the exception. Since this example generates the exception within code on the frame, using the EXCEPTION_RECORD to locate the exception works. However, actual implementations would likely need to use DISPATCHER_CONTEXT instead.
Obviously, this particular implementation doesn't scale well. The ideal solution would involve a single handler that could correctly identify whether an address was guarded and then jump to the appropriate in-procedure handler code for any exception. Higher-level languages have a mechanism for doing this since they can write custom data for the language-specific handler as part of writing UNWIND_INFO. One can imagine writing a table of block addresses and handler addresses that could be referenced at runtime. Presumably macros could be used to implement a similar approach in raw assembly code.
In fact, one doesn't have to completely imagine. If you review Matt Pietrek's article on Win 32 exception handling (linked above), you will notice that he discusses how structured exception handling is implemented. Indeed, it is a table-based approach that extends the exception registration information structure adding things like a scope_table_entry, trylevel, and _ebp. I won't rehash Matt's descriptions of how these work, but I do want to point out that the UNWIND_INFO structure (which is more or less analogous to the Win32 EXCEPTION_REGISTRATION structure) also allows for any custom data (like scope_table_entry) to be appended to the end of the structure. To leverage this feature, you would need to manually register your unwind information via a call to RtlAddFunctionTable rather than using the MASM psuedo-operations.
The other thing that should be mentioned is that in the case where the filter function intends to continue execution when handling an exception that occured in another frame, the stack will need to be unwound. That can be accomplished easily by calling RtlUnwindEx. Be aware, however, that all frames being unwound must be 16-byte aligned and they must have an associated function table entry with correct UNWIND_INFO. If either of these conditions are not satisfied, RtlUnwindEx will throw an invalid stack exception.
So with that in mind, here is a very prmitive example that hopefully will bring at least a few of these concepts to light.
DISCLAIMER: I am not an assembly expert and, as I'm still in the research phase of my project, I have not yet taken time to fully understand the language. The code you see below should not be construed as being good, correct, or proper.
.data
someLocation DWORD ?
.code
ExceptionFilter PROTO
PEXCEPTION_RECORD TYPEDEF PTR EXCEPTION_RECORD
LPVOID TYPEDEF PTR VOID
EXCEPTION_DISPOSITION STRUCT DWORD
ExceptionContinueExecution = 0
ExceptionContinueSearch = 1
ExceptionNestedException = 2
ExceptionCollidedUnwind = 3
EXCEPTION_DISPOSITION ENDS
EXCEPTION_RECORD STRUCT
xcode DWORD ?
flags DWORD ?
precord PEXCEPTION_RECORD ?
xaddress LPVOID ?
cparams DWORD ?
excepinfo QWORD ?
EXCEPTION_RECORD ENDS
CONTEXT STRUCT
P1Home QWORD ?
P2Home QWORD ?
P3Home QWORD ?
P4Home QWORD ?
p5Home QWORD ?
p6Home QWORD ?
flags DWORD ?
MxCsr DWORD ?
SegCS WORD ?
SegDS WORD ?
SegES WORD ?
SegFS WORD ?
SegGS WORD ?
SegSS WORD ?
EFlags DWORD ?
_Dr0 QWORD ?
_Dr1 QWORD ?
_Dr2 QWORD ?
_Dr3 QWORD ?
_Dr6 QWORD ?
_Dr7 QWORD ?
_Rax QWORD ?
; There is more to context, but I just needed rax
CONTEXT ENDS
PUBLIC TestExceptionHandling
TestExceptionHandling PROC FRAME:ExceptionFilter
.ENDPROLOG
GuardTop = $
mov rax, 0
mov rbx, 100
mov [rax], rbx ; AV dereferencing NULL
jmp exit
GuardBottom = $
Handler::
lea rax, someLocation
mov [r8].CONTEXT._rax, rax ; change the context so someLocation gets dereferenced instead of NULL
mov rax, ExceptionContinueExecution
ret
Exit:
ret
TestExceptionHandling ENDP
ExceptionFilter PROC
lea rax, GuardTop ; get the start of the protected block
cmp [rcx].EXCEPTION_RECORD.xaddress,rax ; are we within the start range?
jl Unhandled ; if not bail
lea rax, GuardBottom ; get the end of the protected block
cmp [rcx].EXCEPTION_RECORD.xaddress,rax ; are we within the end range?
jg Unhandled ; if not bail
jmp Handler ; handler code is in the proc
Unhandled:
mov rax, ExceptionContinueSearch
ret
ExceptionFilter ENDP
END