Fix for possible NGen service deadlocks during product installation is coming in the .NET Framework 4
A while back, I wrote a blog post about a scenario that can lead to deadlocks when using synchronous native image generation (NGen) actions in a setup program. I recently heard from Surupa Biswas (the program manager on the NGen team at Microsoft who originally informed me of this issue), and she let me know that her team has implemented a fix in the NGen service that will solve this problem in the upcoming release of the .NET Framework 4. The fix is not available in the .NET Framework 4 beta 2, but will be available in the final release early in 2010. Here is more information about the issue, the fix, and the implications for setup programs that use NGen.
Introduction
In previous versions of the .NET Framework, any setup that includes NGen actions for optional components can potentially cause deadlocks because the NGen service may invoke the installation and native image generation for an optional component, and the NGen service is not fully re-entrant. In the upcoming release of the .NET Framework 4, the NGen team has addressed this potential deadlock by modifying NGen so that it never invokes the installation and native image generation for an optional component.
What will happen at NGen time
When NGen attempts to generate a native image for A.dll, and A.dll has a static dependency on some optional component B.dll that is not yet installed, NGen will generate a partial native image for A.dll (whereas the previous version of NGen would have attempted to install and generate a native image for B.dll, which led to potential deadlocks). This partial native image contains all the native code and data structures that can be generated without using information from B.dll. For example, if a type in A.dll inherits from a type in B.dll or contains some fields of types defined in B.dll, then this type will not be included in the partial native image.
What will happen at execution time
When the user runs a managed application that uses A.dll, the CLR will load the partial native image for it. Any native code and data structures that are included in the native image can be used by CLR in the normal way. If the application tries to use a type or method that is missing from the partial native image, the CLR will notice the missing information, and it will try to generate the data structures or JIT compile the missing methods. Since B.dll is needed to generate the data structures, the CLR will ask the setup program for A.dll and B.dll to install the optional component B.dll. After B.dll is installed, the application can continue running. It is safe for the setup program for A.dll and B.dll to issue both synchronous and asynchronous NGen commands in this case.
What setup should do
Installing B.dll will invalidate the partial native image for A.dll. The partial native image can still be used by any applications that are already running at the time when B.dll is installed, but all applications started later will need a new (complete) native image of A.dll. The setup program should therefore request that NGen recompile A.dll when it installs B.dll. To trigger this recompilation, the NGen team recommend that the setup program issue the command ngen.exe update /queue command at the end of the installation process. This will ensure that all native images that depend on B.dll are updated.