High Memory part 4 - Managed Strings
So the last high memory post, High Memory part 3, focused on native memory that was the problem, let's get back to our normal, managed world. So other then DataTables which we save in the previous post, High Memory continued, what else could cause us to see a large amount of memory being used by .NET?
There are a bunch of other problems that can cause high memory. If most of the memory is being taken up by .NET, then we can look at the output of !sos.dumpheap -stat to see what is taking up most of the memory. One common thing to see is something like:
0:000> !clr10\sos.dumpheap -stat
...
Statistics:
MT Count TotalSize Class Name
...
0x0b80a234 11,685 233,700 System.Web.FileMonitorTarget
0x0d81f344 6,044 241,760 System.Web.UI.WebControls.Style
0x79ba968c 5,948 309,296 System.Collections.Hashtable
0x0f4fa25c 22,641 362,256 System.Web.Configuration.CapabilitiesPattern
0x0f4fa314 19,580 391,600 System.Web.Configuration.CapabilitiesAssignment
0x0f4f578c 4,932 572,112 System.Web.UI.WebControls.DataGridItem
0x00e331e8 24,624 728,648 System.Int32[]
0x79ba8cc8 119,338 1,432,056 System.Int32
0x00e334a8 6,058 1,984,248 System.Collections.Hashtable/bucket[]
0x0c0c8a7c 187,381 2,248,572 System.ComponentModel.EventHandlerList
0x0f4f864c 117,158 2,343,160 System.Web.UI.Triplet
0x0d814bb0 193,637 3,098,192 System.Web.UI.StateBag
0x0d963fbc 206,088 3,297,408 System.Web.UI.StateItem
0x10181dfc 228,298 3,652,768 System.Web.UI.Pair
0x0c0cb334 187,588 3,751,760 System.ComponentModel.EventHandlerList/ListEntry
0x0c14c620 193,747 3,874,940 System.Collections.Specialized.HybridDictionary
0x0c0c2cdc 206,354 4,127,080 System.Collections.Specialized.ListDictionary/DictionaryNode
0x0c0c2be8 193,728 4,649,472 System.Collections.Specialized.ListDictionary
0x79bc3564 188,036 5,265,008 System.EventHandler
0x79ba0d74 238,016 5,712,384 System.Collections.ArrayList
0x00e3292c 600 6,570,828 System.Byte[]
0x00e3209c 276,701 14,448,052 System.Object[]
0x00e3236c 490 17,826,392 System.Char[]
0x0f4f644c 187,416 17,991,936 System.Web.UI.WebControls.TableCell
0x105239f0 948,948 22,774,752 Test.TestDate
0x10559c20 243,243 35,026,992 Test.TestCollection
0x79b925c8 1,188,863 132,204,224 System.String
0x001516a0 122 226,349,940 Free
Total 5,288,099 objects, Total size: 523,832,112
From this, we can see that most of the memory is being taken up by System.String objects. So there are a few ways we can look at the strings. One of my favorite ways is to look at them grouped by size. There is a switch to !dumpheap that that does that. Here is what it looks like in this case:
0:000> !clr10\sos.dumpheap -mt 0x79b925c8 -sizestat
Using our cache to search the heap.
Address MT Size Gen
Statistics:
MT Count TotalSize Class Name
0x00009c40 3 34,044
0x00002710 10 78,292
0x00001388 230 332,184
0x000001f4 78,748 15,744,792
0x000186a0 83 23,806,908
0x000003e8 80,049 42,256,244
0x00000064 1,029,740 49,951,748
Total 1,188,863 objects, Total size: 132,204,212
MT's are the upper bound of size for objects.
0x00000064 = 100
0x000001f4 = 500
0x000003e8 = 1,000
0x00001388 = 5,000
0x00002710 = 10,000
0x00009c40 = 40,000
0x00014c08 = 85,000
0x000186a0 = All above 85,000
This output is a little different, so the column listed at MT is actually the size of the object and then the count and total size in that range. So now we can see that most of our strings are small and there are just a lot of them that are causing all the memory usage. The next steps would be to look at the strings themselves, using !dumpheap -mt 0x79b925c8 (maybe use the -min and -max switches to limit it by size) and look for anything that jumps out. The most common problem would be string concatenation. This is usually seen with html strings where you would see one string that is "<table><tr>" and then the next string is "<table><tr><td>Data</td>" and so on, where each string builds on the next one. This problem is easily fixed by using the System.StringBuilder class. Take a look at the following articles for more information:
How to improve string concatenation performance in Visual C#
How to improve string concatenation performance in Visual Basic .NET or in Visual Basic 2005
Some of you may have noticed the large amount of memory that is being taken up by the Free object. We will address that in a future posting.
Comments
Anonymous
February 13, 2008
The comment has been removedAnonymous
February 15, 2008
TGIF, almost time for the weekend... but before you leave, here is lab 3. Todays debugging puzzle willAnonymous
February 15, 2008
TGIF, almost time for the weekend... but before you leave, here is lab 3. Todays debugging puzzle willAnonymous
February 18, 2008
The comment has been removedAnonymous
February 19, 2008
I was reading this post about spam ping-bots from OldNewThing and it makes me a bit mad because theyAnonymous
May 08, 2008
The -sizestat option does not work, I've tried the various versions of sos.dll that I have an none work, can you offer any help?Anonymous
May 08, 2008
Rick, It should be in the sos.dll that is in the clr10 subdirectory installed with the debugger package. This only works with .NET 1.0 and 1.1. I am currently working on getting something out for 2.0 so keep checking this blog for more information. You can also add what features you want to see on this post: http://blogs.msdn.com/tom/archive/2008/03/25/asp-net-and-sos.aspxAnonymous
May 13, 2008
It's not in the clr10sos.dll (File Version: 6.9.3.113) :(Anonymous
May 14, 2008
Hmm... so if you run !clr10sos.help dumpheap you don't see the -sizestat listed as an option? If not, hopefully it will be in the next release.Anonymous
June 02, 2008
What's the File version you have for clr10sos.dll?Anonymous
June 03, 2008
my clr10sos.dll is 6.10.0.151 so hopefully the next drop will have that update it in.