C# compiler or Visual Basic .Net compilers fail with error code -1073741502 when generating assemblies for your ASP.net site

[January 2018 Updates:] This article applies to both Windows 2008R2 servers and Windows 2012R2 servers, running IIS 7.5 and 8.5 respectively. If you encounter the symptoms described below, please go directly to the solution update described at the end of the article.

A few days ago I had a support call where an ASP.net application deployed to a Windows 2008 R2 server and running on IIS 7.5 was crashing from time to time. When a page from the application crashed, a 500 error code was returned with an error message similar to the following:

Compiler Error Message: The compiler failed with error code -1073741502.

The error is then followed by the command line with which the .Net compiler was launched. This can be a very daunting error as it can appear and disappear randomly, and recycling the app pool or restarting IIS will not help. If you just need the error solved, go directly to the solution part of the article, if not, keep reading for the detailed explanation.

So what actually goes on:

ASP.net applications can be deployed in several modes to an IIS server (I discuss this in more detail in the following video: https://linqto.me/IISArchP3 ), but basically each source code file in an ASP.net application (be it a .cs – C# file or a .vb – VB.Net file) will have to be compiled before it can be executed on the server. Generally this occurs when a request for that particular par or resource comes in from an end user.

Suppose we have an application that has a page called Default.aspx. The page code behind is written in C# and is present in a file called Default.aspx.cs. When the first request for this page comes in from the very first user to hit the site, the .Net Runtime will notice that the page is not yet compiled and will request that the page be compiled – the C# code be transformed into a .dll that can be executed - .

When the page has to be compiled, what will happen is that the w3pw.exe (worker process) that hosts the ASP.net application will request the activation of the .Net compiler (csc.exe – C# compiler for C# code or vbc.exe for VB.net code). You can see a detailed architecture diagram of all IIS components and how they interact in this video: https://linqto.me/IISArchP1 and a detailed explanation of the inner workings of the worker process (w3wp.exe) in this video: https://linqto.me/IISArchP2 .

The .Net Runtime will then pause the execution of the request pending the execution of the compiler process (csc.exe or vbc.exe). You can actually see the compiler activating if you look at the Task Manager of your server when the first request comes in. The compiler will run with the same credentials (account) as those used for the application pool. By default this is Application Pool Identity is used as of IIS 7.5, but you can configure your own account in the 'Advanced Properties' of the application pool.

Should the compiler process encounter an error (be it a compile error that you introduced in your code or something else) it will then transfer the error message back to the worker process, which will just issue a 500 (internal server error) status code for the particular request back to the connecting client. So we now know that the error is coming from the csc.exe process that runs with the same credentials and in the same session with the w3wp.exe. Now to the error message that is raised in the particular case.

Looking at the return code value, the first thing to do is to convert from base 10 to base 16 – or HEX, and we can do this using the debugger or even the windows calculator:

.formats -0n1073741502
Evaluate expression:
Hex: c0000142
Decimal: -1073741502
Octal: 30000000502
Binary: 11000000 00000000 00000001 01000010
Chars: ...B
Time: ***** Invalid
Float: low -2.00008 high -1.#QNAN
Double: -1.#QNAN

Then using a tool to lookup the hexadecimal error code we can see the following:

!error c0000142

Error code: (NTSTATUS) 0xc0000142 (3221225794) - {DLL Initialization Failed} Initialization of the dynamic link library %hs failed. The process is terminating abnormally.

Hence the csc.exe process terminated because of a Dll initialization failure. This will most likely happen when the system is low on what is called Desktop Heap memory. This is a special kind of memory that is allocated at boot time for each of the sessions opened by an account on the machine. Hence, all processes that are running using the credentials of that particular account will share resources from the Desktop Heap that is allocated for that account's session.

Should there be too many processes running under the same account's credentials, the desktop heap can be close to exhaustion, so resources might not be available to dlls that need to load inside a particular process. Here is an article on how to resize the Desktop Heap segments: https://blogs.msdn.com/b/ntdebugging/archive/2007/01/04/desktop-heap-overview.aspx – but what I recommend is a different solution.

Solution:

Now that we know that this error is generated by Desktop Heap exhaustion, a simple solution is to create a dedicated account for the application pool for which we are seeing the error. Hence, the worker process and the csc.exe process will have the entire Desktop Heap segment available account to themselves, hence bypassing the resource consumption problem.

As a final note, I would like to underline that the default configuration for IIS application pools, in which each pool dynamically is allocated its own identity (account) by IIS when application pool starts, bypasses the problem as well, since each application pool, as well as the compiler instances it needs to launch, do not need to compete for the scarce resources of a shared desktop heap.

Solution Update - January 2018

The crash of the compiler is being caused by Desktop Heap exhaustion for the Desktops associated with the Non-Interactive Window Station. This is causing the either user32.dll or kernel32.dll to fail to initialize properly. The blog article mentioned above goes a long way to explain what the desktop heap is and what it is used for. Suffice to say, that all IIS worker processes and, in general, all Windows services, are running in Desktops that pertain to the Non-Interactive Window Station. This means, that graphical interface generated from code in an IIS worker process or a Windows service (like message boxes) will not show up in a user's Desktop, which pertains to the Interactive Session.

By default, the size of each desktop heap for a Desktop that belongs to the Non-Interactive Window Station is defined via registry, using the following key:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\SubSystems\Windows

The value of the Windows registry key is the following:
%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16

The shared section part (in bold above), is the part that describes the Desktop Heap sizes in KiloBytes. We are interested in the third value (of 768) which is the size of the Desktop Heap for the Desktops in the non-interactive Window Station. If changing the account for the app pool does not solve the issue, increasing the size of the heap will. I recommend adding 4 MB more memory to this heap, by augmenting the value from its default of 768 to 4864 (786 + 4096 KB). Once you make this change using the registry editor, you will need to restart your operating system (reboot Windows) to allow the change to be taken up.

After modification the registry key value should look like:

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,4864 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16

With a larger Desktop Heap available, the compiler will now have the needed resources to initialize, and will be able to perform the compilation of ASP.net pages.

By Paul Cociuba – ASP.net Engineer

Follow what I read via LinqTo.me

Comments

• Anonymous
  October 15, 2012
  Really like that you are providing such info on .net being enrolled in .NET freshers training at www.wiziq.com/.../57-fresher-training-projects i really thank that you are providing such information.Thanks a lot.

• Anonymous
  October 15, 2012
  Really like that you are providing such info on .net being enrolled in .NET freshers training at www.wiziq.com/.../57-fresher-training-projects i really thank that you are providing such information.Thanks a lot.

• Anonymous
  October 15, 2012
  @sarabjeet: Happy to see this helps out! Paul

• Anonymous
  December 30, 2012
  Thanks for this

• Anonymous
  May 29, 2013
  Would a permissions issue be likely to cause this, as well?

  • Anonymous
    April 23, 2016
    In my case a new application pool was created, the identity was set to an account with admin rights on the server which compiled the website and then set back to ApplicationPoolIdentity
    • Anonymous
      August 25, 2016
      This sounds like a workaround to the problem. By elevating privileges of the application pool identity to an admin account, all child processes, including csc.exe will be launched with that account privileges. Hence they will have access to anything on disk. You should try and avoid elevation as much as possible. Also, the next time you have to compile, you might bump into this one again.
      • Anonymous
        May 03, 2017
        The comment has been removed

• Anonymous
  January 15, 2014
  Thanks -  it was enough for me to stop and restart the application pool which is unique to the application.

• Anonymous
  April 23, 2016
  This blog pushed me in the right direction to fix a critical production application in my company! Thank you Paul!

  • Anonymous
    January 03, 2017
    Happy to be of service Christian.