Detours
Still on the topic on knowing what DllRegisterServer does without impacting the system. Before I found RegOverridePredefKey, I was thinking about hijacking all the registry APIs so that I can just record them without actually making any system change.
Apparently, I need a way to safely detour those registry APIs to my own implementation. I was going to roll my own, then I found this:
Overview
Innovative systems research hinges on the ability to easily instrument and extend existing operating system and application functionality. With access to appropriate source code, it is often trivial to insert new instrumentation or extensions by rebuilding the OS or application. However, in today's world systems researchers seldom have access to all relevant source code.
Detours is a library for instrumenting arbitrary Win32 functions on x86, x64, and IA64 machines. Detours intercepts Win32 functions by re-writing the in-memory code for target functions. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary.
Detours preserves the un-instrumented target function (callable through a trampoline) as a subroutine for use by the instrumentation. Our trampoline design enables a large class of innovative extensions to existing binary software.
We have used Detours to create an automatic distributed partitioning system, to instrument and analyze the DCOM protocol stack, and to create a thunking layer for a COM-based OS API. Detours is used widely within Microsoft and within the industry.
I love Microsoft Research:)