ASP.NET Memory: If your application is in production… then why is debug=true
Statement
“Ensure that the debug="false" on the <compilation> element in the web.config file of each and every ASP.NET application on the server. The default during development is "true" and it is a common mistake to allow this development time setting to find its way onto production servers during deployment. You don't need it set to true in production and it often leads to memory overhead and inefficiencies.”
What problems does leaving debug=true cause?
There are three main differences between debug=true and debug=false:
- ASP.NET Timeouts
- Batch compilation
- Code optimization
ASP.NET Timeouts
When debug is set to true, asp.net requests will not time out. This is to allow you to debug with visual studio at your own pace without having to worry about the requests suddenly disappearing. Of course in a production environment timeouts are crucial to avoid for requests to be stuck indefinitely, so this is reason #1 to make sure debug is set to false when the application is deployed into production.
Batch compilation
In short, when debug=true, we don’t batch compile, when debug=false we do…
What does this mean?
When an aspx, asax, or ascx page is first requested it gets compiled into an assembly. This assembly has a name like 3ks0rnwz.dll or similar (8 characters) and stores the class for the actual ascx, asax, or aspx page (not the code behind). The assembly goes into a folder in the C:WINDOWSMicrosoft.NETFrameworkv1.1.4322Temporary ASP.NET Files with the same name as the application.
The code behind class gets compiled into the main dll for the assembly, and it along with all the other dlls in the applications bin directory get shadow copied to the Temporary ASP.NET files.
Back to the 3ks0rnwz.dll… If we have debug=true, we create one dll per aspx, asax, or ascx page and this dll is compiled in debug mode, so if you have 100 web pages, you will have 100 assemblies, and they are generated as the pages are requested.
If we instead have debug=false, we batch compile, which means that the first time you request any page in your application, we compile the whole batch into one big assembly. This is a truth with some modification. The user controls (ascx pages) are compiled into a separate assembly from the aspx pages and the aspx pages are compiled in groups based on what other files (read usercontrols) they include. The global.asax is also compiled separately. And batch compilation occurs on a directory bases, meaning that if your application has subdirectories, the subdirectories are compiled separately to avoid for example name clashes, as it is valid to have two aspx pages with the same name in different directories. But all in all, instead of 100 dlls, you might end up with 3 or 4.
Ok, big deal? It’s the same code so the size of the combined assemblies shouldn’t much differ from the size of the individual assemblies right? Truth is, there probably isn’t an enormous difference. But… and this is a big but… there is overhead for each dll, and if the dll is compiled in debug mode there is overhead for items needed for debugging, and … last but not least (in fact probably most important), the assemblies won’t be laid exactly side by side, so with a large number of assemblies you start fragmenting the virtual address space making it harder and harder to find large enough spaces to store the managed heaps, potentially causing out of memory exceptions.
One caution even if you have debug=false, is that if you go in and change something in one of your aspx pages, this page will have to be recompiled, but this doesn’t cause an appdomain reload so the whole application is not batch compiled again. This has the effect that the page will now get recompiled separately and get its own dll, so don’t change your aspx pages on a live server too often.
There is a setting in machine.config determining how many recompiles are allowed before the app domain restarts, by default it is set to 15, so after 15 recompilations the app domain will restart, just as it would if you touched the web.config or touched the bin directory.
Code optimization
In order to be able to step through code line by line the JITter can’t really optimize the code which means that your debug dlls will be less performant than if they were compiled in release mode.
So as you can probably figure, there is a large benefit to having debug=false in production…
How can you identify it in a memory dump?
To find out if any of the applications on your server run with debug=true you can run a nifty command in sos.dll called !finddebugtrue which will list out all applications where debug=true in the web.config, now how easy is thatJ
0:016> !finddebugtrue
Debug set to true for Runtime: 61b48dc, AppDomain: /MyDebugApplication
Debug set to true for Runtime: 1f50e6d8, AppDomain: /MemoryIssues
Total 16 HttpRuntime objects
And to find out if you forgot to compile some of your assemblies in release mode run !finddebugmodules
0:016> !finddebugmodules
Loading all modules.
Searching for modules built in debug mode...
MyDebugApplication.dll built debug
MemoryIssues.dll built debug
fl4sq-9i.dll built debug
wepr1t3k.dll built debug
r9ocxl4m.dll built debug
zmaozzfb.dll built debug
Done Seaching
The dlls above with weird 8 character names are the dlls generate when JITing the aspx pages, so they will go away when debug=false.
Oh, before I forget… when you change from debug=true to debug=false it is a good idea to clean out your Temporary ASP.NET files for this application so you don’t have some old junk in there causing it to still not batch compile.
In ASP.NET 2.0 there is a switch that can be turned on in machine.config that turns off all debug=true, so in 2.0 applications you can do this directly without worrying about finding out which applications do and don’t have it.
<configuration>
<system.web>
<deployment retail=”true”/>
</system.web>
</configuration>
If you want some more goodies about debug=true, read ScottGu’s blog post about it https://weblogs.asp.net/scottgu/archive/2006/04/11/442448.aspx
Comments
Anonymous
April 12, 2006
Recently my colleague Doug wrote a nice post on Nine tips for a healthy "in production" ASP.NET application....Anonymous
April 13, 2006
if yo ugo back and change debug=true to debug=false wont you get a compilation error? do yo need to restart iis after doing this?Anonymous
April 13, 2006
Another slight app startup performance hit you'll have if you set debug to false is that (and i'm guessing here) when the ASP.Net app creates a dll for each aspx page, its not finding a unique base address in memory for each dll. So when the AppDomain loads it will have to spend time rebasing each and every dll into an empty spot in memory. if you only have 3-4 dlls, its much less of a hit.Anonymous
April 15, 2006
Gregor,
You should not get an error message just by changing from debug=true to debug=false, but to avoid having some dlls batch compiled and some not i would recommend deleting the temporary asp.net files when you next IISresetAnonymous
April 16, 2006
Ok &ndash; I admit, I have done this, you take the web.config from development, it gets rolled up to...Anonymous
April 18, 2006
Uno degli errori più comuni quando si effettua il deployment in ambiente di produzione di una applicazione...Anonymous
April 24, 2006
We did get an error for a UserControl that it reported it could not load the FileName_ascx class due to multiple versions in the Temp ASP.NET folder.
We identified that we had two user controls with the same file name in diferent Folders of the same Web App. The also had diferent namespaces and never through this exception until we set debug="false". We even wiped the Temp ASP.NET directory clean on an IISreset.
The only way we could fix the error, was by renaming the ascx file of one of the two.
Is this correct...? Was there a better way to fix this?
BTW...
[KissUpText]
Tess, your posts have been very helpfull to our development team, and we really appreciate all the information you have given away.
[/KissUpText]Anonymous
April 24, 2006
Hi Robbie,
Thanks for the nice comment:)
I am assuming that you are getting "CS1595: 'UserControls.WebUserControl2' is defined in multiple places; using definition from 'c:WINDOWSMicrosoft.NETFrameworkv1.1.4322Temporary ASP.NET Filesusercontrols293a1a4bdbb2d387cisxatg3.dll'
" or similar.
The problem basically occurrs if you are using src rather than CodeBehind and your cs or vb files contain a definition for exactly the same class in exactly the same namespace. The error is really the same as what you would get if you tried to compile a dll with another class defined twice in the same namespace.
The reason i am saying it happens when you use src is because if you would use CodeBehind you would have gotten an error at compile time.
If the usercontrols are really the same I would avoid creating a copy, and instead using the one from the other folder. If they are different I would either give the different names if possible, and if not, make sure that the source classes are in different namespaces, such as ProjectName.FolderName.MyUserControl
The reason you are seeing it now and not before is because you are now batch-compiling everything into one dll.
Hope this helps.Anonymous
April 25, 2006
Thanks Tess,
Sorry, I should have include the exception message:
CS1595: '_ASP.BrokerInformation_ascx' is defined in multiple places; using definition from 'c:WINDOWSMicrosoft.NETFrameworkv1.1.4322Temporary ASP.NET Filesrootf4fb459b4deb68eb1huxv2vg.dll'
And actually, we are using the CodeBehind attribute:
-------------------------------------------------------------------------
UserControl #1:
<%@ Control Language="c#" AutoEventWireup="false" Codebehind="BrokerInformation.ascx.cs" Inherits="WAB.Websites.GA.UserAdmin.UserControls.BrokerInformation" TargetSchema="http://schemas.microsoft.com/intellisense/ie5"%>">http://schemas.microsoft.com/intellisense/ie5"%>
CodeBehind #1:
namespace WAB.Websites.GA.Admin.UserControls
{
/// <summary>
/// Summary description for BrokerInformation.
/// </summary>
public class BrokerInformation : UserControlBase
...
-------------------------------------------------------------------------
UserControl #2:
<%@ Control Language="c#" AutoEventWireup="false" Codebehind="BrokerInformation.ascx.cs" Inherits="WAB.Websites.GA.Admin.UserControls.BrokerInformation" TargetSchema="http://schemas.microsoft.com/intellisense/ie5" %>
CodeBehind #2:
namespace WAB.Websites.GA.UserAdmin.UserControls
{
/// <summary>
/// Summary description for BrokerInformation.
/// </summary>
public class BrokerInformation : UserControlBase
...
(this is how it looked when the error was happening)
Also, yes they are different controls, and we have since renamed the #2 control AdminBrokerInformation.ascx.
So we are no longer receiving the error, but it looks to me that within a ASP.NET (1.x) application, all aspx and ascx files must be named unique (and not in the FQCN meaning of unique), because they all end up under the same "_ASP.*" namespace.
please let me know if:
Assert.IsTrue(MyCommentAbove.IsCorrect);Anonymous
April 28, 2006
Tess explains in detail why you should never run a production site with debug enabled:
ASP.NET Memory:...Anonymous
April 28, 2006
I always knew that debug=true in the web config of an ASP.NET project was bad karma.&nbsp; However, this...Anonymous
April 29, 2006
Every ASP.Net developer knows about this setting, or at least should. But what happens when you forget...Anonymous
May 03, 2006
Man lernt doch nie aus. Hier steht warum es uerst schlecht ist, in der web.config einer ASP.NET-Applikation das Flag "debug" auf "true" zu lassen sobald die Applikation produktiv eingesetzt wird.
Weiter finden sich hier Nine tipsAnonymous
May 03, 2006
Warum sollte man&nbsp;debug=false setzen?
http://blogs.msdn.com/tess/archive/2006/04/13/575364.aspxAnonymous
May 05, 2006
The following links to .NET resources have been collated over time with the assistance of colleagues.&nbsp;...Anonymous
May 07, 2006
The comment has been removedAnonymous
May 07, 2006
The comment has been removedAnonymous
May 10, 2006
Right, I know it's a bit controversial to leave your production server running with debug symbols intentionally. However, Considering that the sites I usually build, are cached (public sites with most page requests being cache hits), I think the extra error info justifies the overhead (which i assume is 0 for a cache hit).Anonymous
May 10, 2006
Рекомендую почитать блог &laquo;If broken it is, fix it you should&raquo;. Детально рассматриваются различные...Anonymous
May 15, 2006
The comment has been removedAnonymous
May 21, 2006
One of the things you want to avoid when deploying an ASP.NET application into production is to accidentally...Anonymous
July 22, 2006
I disabled the debug on the production but suddenly the home page stop responding my requests (there is login button when i try to login the page just post back without login to the system), what happened??!!Anonymous
July 26, 2006
Only thing I can think of is that disabling debugging generated some type of exception, perhaps because of duplicate class names or similar. I would attach a debugger and try to log in to see if something like that is happening.Anonymous
August 03, 2006
We set debug=false in our production environment, and this caused the site to crash due to CS1595 [user control] is defined in multiple places. IISReset + wiping the Temp ASP.NET files clears up this error temporarily, but it reoccurs again later if a change is made to the web.config. I've seen all the threads about multiple DLLs, Src vs. CodeBehind, compiler flags, but this is not the cause in our case. We never had this issue until setting debug=false.Anonymous
August 03, 2006
Hi Jeremy,
This can also happen if you have multiple virtual directories pointing to the same physical path, in which case there can be a problem like this during re-compilation of the page, because the old dlls don't get removed.
It only happens if batch compilation is set to true, which is why you only see it when debug=true.
You can use this to turn batch compilation off while debug is still set to false, but then you get one dll per page
<compilation defaultLanguage="c#" debug="false" batch="false" />
HTHAnonymous
August 07, 2006
PingBack from http://spicycode.com/articles/2006/08/08/how-to-win-friends-and-improve-app-performanceAnonymous
August 07, 2006
PingBack from http://spicycode.com/articles/2006/05/15/how-to-win-friends-and-improve-app-performanceAnonymous
August 08, 2006
Question regarding debug settings in VS 2005... There's a new setting for debug info, pdb-only. How does this compare to full debug info?Anonymous
August 08, 2006
The comment has been removedAnonymous
September 30, 2006
Continuing from my previous post on common causes for memory leaks, remote debugging is another ASP.NET...Anonymous
December 19, 2006
It sounds like you should always run a production site in Debug=false in the web.config what about running that same site using Dlls that where compiled in debug mode. Do you still run into issues with Debug compiled Dlls?Anonymous
December 19, 2006
Hi Kevin, The timeout and non-batchcompilation is specific to web apps running with debug=true, but you do run into some of the other things i.e. the code is not optimized and there is an overhead on the loader heap for debug true.Anonymous
January 11, 2007
PingBack from http://dotnetdebug.net/2006/04/15/tips-for-a-healthy-aspnet-application-in-production/Anonymous
January 17, 2007
Very good informative article.Anonymous
February 11, 2007
Hi, I am setting <compilation debug="false"> in Web.config to control my compilation. I am using aspnet_compiler to build my site. Now, my question sounds very simple but I can't seem to find a clear answer to it: does it matter whether I set <compilation debug> to "false" BEFORE or AFTER running aspnet_compiler? I thought that I had figured out that it was necessary to set debug="false" before compiling (makes sense?), but now I am just confused. If it is important to set debug="false" BEFORE running aspnet_compiler, then I have a follow-up question: why do Visual Studio Web Deployment Projects set it AFTER?! (Have a look at the AspNetCompiler target in Microsoft.WebDeployment.targets). Thanks v. much.Anonymous
February 15, 2007
The comment has been removedAnonymous
February 18, 2007
the debug setting in web.config is only used for jit compilation at runtime so it should be set when you publish your applicationAnonymous
February 18, 2007
the debug setting in web.config is only used for jit compilation at runtime so it should be set when you publish your applicationAnonymous
March 02, 2007
PingBack from http://andrenobre.wordpress.com/2007/03/03/debug-or-release/Anonymous
March 17, 2007
Hi, Tess You wrote: "i would recommend deleting the temporary asp.net files when you next IISreset". "Deleting" method integrated in Microsoft(c) Windows, isn't it? Or "delete", in this case, means lowlevel deleting files from disk? Thanks...Anonymous
March 18, 2007
Delete as in deleting the files from disk...Anonymous
March 29, 2007
Continuing from my previous post on common causes for memory leaks, remote debugging is another ASP.NETAnonymous
June 22, 2007
Great it is as easy as just typing !finddebugmodules. Is it really?????? I just learned with lot of disappointment that SOS dll does not have this. I would love to debug and fix if only better information is available to end devolopersAnonymous
June 22, 2007
Also why if retail=true in machine.config still finddebugmodule lists some as debug modules (why doesnt retail=true override everything). why you provide a feature anbd it does not workAnonymous
June 24, 2007
The comment has been removedAnonymous
September 05, 2007
Changing Debug= "false" in the Web.Config file is causing HTML code generation of an OnClick Event to not generate from the code behind. HTML code is generated for the OnClick Event when Debug= "true" in the Web.Config file. Here are the code behind .vb statements: CType(Me.FindControl("btnPrint"), ImageButton).Attributes.Add("OnClick", "return PrintPage();") CType(Me.FindControl("btnSearchDB"), ImageButton).Attributes.Add("OnClick", "return Search('DivSearch');") And here are the HTML generated statements: <TD class="SDToolBarButton"><asp:imagebutton id="btnSearchDB" ImageUrl="..Imageslens.gif" runat="server" ToolTip="Search Database"></asp:imagebutton><asp:imagebutton id="btnSearchDBDisabled" ImageUrl="../Images/lensDis.gif" runat="server" ToolTip="Search Database" Visible="False" CssClass="SDToolBarDisabledButton"></asp:imagebutton></TD> <TD class="SDToolBarButton"><asp:imagebutton id="btnPrint" ImageUrl="..ImagesPrint.gif" runat="server" ToolTip="Print Screen"></asp:imagebutton><asp:imagebutton id="btnPrintDisabled" ImageUrl="..ImagesPrintDis.gif" runat="server" ToolTip="Print " Visible="False" CssClass="SDToolBarDisabledButton"></asp:imagebutton></TD> It's definitely broke, any ideas on how to fix it?Anonymous
September 09, 2007
What is sos.dllAnonymous
September 09, 2007
sos.dll is an extensions for windbg.exe that allows you to debug managed applications. (Windbg.exe is normally native only) it comes with the framework and you can find it in the framework directory.Anonymous
October 08, 2007
Hi, I read your and Scott's blog, and can't find a solution to the HTTPS problem. That is when using asp:menu control, users in IE6 get a "security" warning on EVERY mouse over action, as it's trying to call the WebResource.axd file. Is this a symptom of debug=true, or something else? RenataAnonymous
October 08, 2007
It is not a symptom of using debug=true, assuming that you get "This page contains both secure and non-secure items. Do you want to display the non secure items" It has to do with the menuitem populating the context menu from an HTTP resource. Check this post out on how to fix it... http://blogs.msdn.com/jorman/archive/2006/02/06/nonsecure-items-message-using-menu-control-over-ssl.aspx Thanks TessAnonymous
October 19, 2007
When are the commands (!finddebugmodules and !finddebugtrue) going to be available for the ASP.NET 2.0?Anonymous
October 21, 2007
I wish I knewAnonymous
October 24, 2007
OK, I'm in a debate and I need to "see" and "touch" what exactly occurs when debug="true" in a web application project. Another developer building an ASP.NET 2.0 Web Application Project, claims that that debug="true" was causing a page to run slow. The page has a code that does a file upload. I put together a sample web application project(note, not an asp.net web site), and using build scripts, build and deploy to two different virtual directories, one with debug="true" and one with debug="false". Poking around the asp.net temp files directory I found the munged code file plus the assembly that was generated. Using Reflector, Beyond Compare, and ILDASM, there are no differences in the output. My question is, is the debug="true" only something that asp.net websites worry about and not web application projects? if not, then what does debug="true" actually do? because I can't see where it stuck in anything more or less than when it was set to "false". TIA, ChrisAnonymous
October 24, 2007
HI Chris, I dont think you will find anything else either, because i dont know how debug=true or false would affect a fileupload. Not exactly sure what you mean by web app projects vs. asp.net web sites but assuming you mean "building the app in vs.net" vs. "compilation that occurrs at runtime" then the answer is that building a web app in vs.net (in 2.0) is mostly a sanity check, to my knowledge no actual code is being compiled (unless you have separate classes in the project). The only thing that matters is debug=true/false, since this affects how the pages get jitted (along with all the other stuff mentioned in the post) I do seem to remember though that if you set the release mode to debug or release that will change the debug setting in web.config, but I can't say for sure.Anonymous
October 28, 2007
Hi Tess, I can not find !finddebugtrue and !finddebugmodules commands in SOS.dll for ASP.NET 2.0. It seems that SOS has fewer commands in this version of .NET framework. How we can see which modules are built with debug="true" using some manual steps or other commands?Anonymous
October 28, 2007
I guess that would be the lm command. Thanks anyway :)Anonymous
January 06, 2008
These tips are reasonably well-known and have been blogged by others. However, considering how oftenAnonymous
May 21, 2008
that was fantastic.i though before that making debug = false will have effect even after we compile the app.Anonymous
May 21, 2008
it does, debug=false is for the JITingAnonymous
January 13, 2009
The comment has been removedAnonymous
April 02, 2009
I did machine.config change on WFEs in MOSS 2007. I am not sure if it's a good idea do it in MOSS but it seems to have made my site stable. Granted it may impact MOSS logging, but if I have to choose stability over functionality, stability takes the cake :).Anonymous
June 07, 2009
Can u explain about sos.dll & how to use the !finddebugtrue... commands.Anonymous
June 29, 2009
Publishing a application compiles it to assemblies. Do we still need debug="false" in the web.config in this scenario. I would like your advice on the changes to this in .net 2.0.Anonymous
June 29, 2009
By default debug=false in 2.0. Wether you need to set it to true or not depends on if your app is pre-compiled or not, but really, there is no reason for not turning debug=true in production.Anonymous
July 15, 2009
Yes, but what about the ajaxtoolkit .dlls that have debug set to true?Anonymous
December 22, 2009
Hello Tess Sorry to resurrect this old zombie issue, but ... I notice in your article you make no mention of .asmx web services. Does this setting have as big an effect on these too? I'm trying to investigate a performance issue on a middle-tier web server that exclusively hosts .asmx web services which are implemented as C# dll's. I think what I'm not sure about :D is whether this setting affects the JIT/runtime compilation of the web service's MSIL objects (i.e. the dll's) and therefore presumably will affect the performance/memory usage of these web services too, or is it just the assembly of those web-app pages and 'code-behind' *.aspx.vb/vc ... er ... things. Hopefully you can make a sensible question out of this :) thanks TimAnonymous
January 06, 2010
Do you have any general benchmarks on how much smaller the memory footprint can be if debug mode = false? Are we talking 50% more memory, twice as much memory, or more? I understand that the answer is "It depends on your app", but do you have any rules of thumb. We've got a pretty complex app with hundreds of thousands of lines of code. We would prefer to keep the debug info in there because we're concerned about the effort to retest the app and side effects it would have on our support team, who are used to using the debug info to resolve customer cases. On the other hand, our app has a huge memory footprint. If the compile in release mode is going to reduce it by 50% or more, then we may need to look into it. Thanks in advanceAnonymous
January 07, 2010
Hi Krister, There is really no good way to say. Picture this... if you have 20 pages = 20 assemblies, and then your app allocates 1 GB worth of memory for various datasets etc. then the overhead for the debug data for the 20 assemblies would be negligeable. If on the other hand you have 1000 assemblies/pages and your app is otherwise frugal with memory usage then the overhead is not so negligeable so it is really impossible to say. Even on a specific page/assembly you can't give a percentage, and it also depends on if the page is changed during the app lifetime. The best thing is probably to test with debug=true or debug=false and see. But more importantly, with debug=true you will not take advantage of timeouts or other code optimization and that is really a bigger issue than the memory usage. What type of debug info does your support team use that would not be available if you have debug=false? debug.write output for example would still be present...Anonymous
January 07, 2010
Tim, Yes, it applies to web services as well. Web services (asmx) is basically a subset of asp.netAnonymous
July 27, 2010
The comment has been removedAnonymous
August 03, 2010
First I thought that debug=true would override this but after looking at this in more detail both in dumps and in documentation I found that retail=true disables the debug=true setting, so if retail=true then debug=true should not be in effect. Thanks, TessAnonymous
August 03, 2010
Thanks Tess for the response....we have an issue where the machine.config has retail=true yet when debug=true is set on the web applications our performance goes out the window. If we set it to false (which why wouldn't we on production anyway?) it goes normal. Our issue has only really been that it slips up there in dpeloyments some times and the last time it got us we put in a critical support ticket with MS who turned around and said you have debugging enabled in production and we see in your dumps that is eating a lot of your resources. Which started the debate on retail=true and debug=true. It seems to me there are a few gotchas still out there and possibly some further digging to what really changes when both these settings are true versus retail=true and debug=false.Anonymous
September 04, 2010
If debug=true you will NOT get time-outs no matter what the setting is for retail in machine.config. That is because the ASK.NET Deadlock detection mechanism literally looks in web.config and if denug=true it theoretically runs for an infinite amount of time so that you have more than enough time to attach a debugger.Anonymous
January 26, 2011
Hi Tess, Thank you for your posts. I would not have solved my application's memory issues without this particular post and all the debugging tips throughout your blog. I've really learned a lot!Anonymous
May 09, 2011
how do i run a a nifty command in sos.dll ?Anonymous
April 24, 2012
The comment has been removedAnonymous
July 05, 2012
Hi Tess, This article is a great read. We have a web application project and use custom http handlers. It has a File import feature to import data from various sources. My question is does this affect custom http handlers? Are the custom http handlers compiled at runtime?Anonymous
May 12, 2015
The comment has been removed