CLR Profiling API

I think CLR Profiling API is one of the coolest things in CLR. Here is a nice article I just found on msdn about Whidbey Profiling APIs. Note that Everett Profiling API and Whidbey Profiling API are completely not compatible. You have to explicitly implement ICorProfilerCallback2 interface to make profiling work on Whidbey.

Profiling provides the possibility to do runtime tracking, tracing and assembly rewriting. It is exactly what a tester would like to do in many aspects to test managed projects.

In order to show some ideas behind this, I will introduce one of our test tools here.

We implemented a Just-In-Time LCG converter tool based on Profiling API's ability to do method instrument. It was always a headache to test LCG because it is a new kind of method, completely dynamic, and touches nearly every aspect of the runtime. The only way to test it well is to leverage our existing tests which already covered nearly all aspects of CLR.

We implemented a DynMethodConverter tool based on Reflection. The tool loads the assembly and based on the methodinfos, it converts them to the dynamic methods, linked them up and invoke on the entry point. Everything looked good at the beginning but we came to many dead problems. The tool only worked on simple, verifiable test libraries. Tests with unmanaged entry points (interop tests, hosting tests), unsafe tests (cannot pass security checks while loading) and tests with drivers will have a difficult time to be adopted under the tool.

Then we looked at the possibility of solving this problem using Profiling API.

When the profiling is turned on, the JITCompliationStarted function gets called. We took that chance to change the method body to call on a DynamicMethod delegate. One tricky part is that LCG has to be created in managed code and Profiling API cannot have any managed stack on top of it. We worked around this issue by starting another thread in the JITCompilationStarted function; invoke managed code that will create the dynamic method there. We also cached that thread id in a global hashtable, so that when that managed code hits JITCompilationStarted function again, we can quickly know that we will have to do nothing there. On the managed side, it will be very natural to use token resolution APIs to get the calling method and leverage all the Reflection structures to create a dynamic method.

This tool can be very easily adapted to the automatic testing environment. We only need to turn the profiler on and off, register the necessary managed assembly as needed. It also solved all the previous limitations of tools based on Reflection.

Hope this sample helps you when you are designing your testing tool.