Share via

Tracking down MmSt paged pool usage

  • Article

 

A trend that I’ve noticed recently are cases involving paged pool depletion with high MmSt tag usage that remains after trying KB304101 (PoolUsageMaximum). These pool allocations are used by the memory manager for section object prototype PTEs. There are generally only two options when this happens: 1) upgrade to a 64-bit platform, or 2) reduce the size of the volumes. But we may want to know what mapped files are using this memory. Here is how it can be done. Start with !memusage.

 

5: kd> !memusage

 loading PFN database

loading (100% complete)

Compiling memory usage data (99% Complete).

             Zeroed: 19073 ( 76292 kb)

               Free: 0 ( 0 kb)

            Standby: 1468824 (5875296 kb)

           Modified: 368 ( 1472 kb)

    ModifiedNoWrite: 1927 ( 7708 kb)

       Active/Valid: 605772 (2423088 kb)

         Transition: 0 ( 0 kb)

                Bad: 0 ( 0 kb)

            Unknown: 0 ( 0 kb)

              TOTAL: 2095964 (8383856 kb)

  Building kernel map

  Finished building kernel map

Scanning PFN database - (100% complete)

 

Following this you will see the list of mapped files and their control areas.

 

  Usage Summary (in Kb):

Control Valid Standby Dirty Shared Locked PageTables name

…

8c62a638 1108 945868 3064 0 0 0 mapped_file( $Mft )

…

 

The control area is the address at the far left and has a Segment field that contains the total number of PTEs.

 

5: kd> dt 8c62a638 _CONTROL_AREA Segment->TotalNumberOfPtes

nt!_CONTROL_AREA

   +0x000 Segment :

      +0x004 TotalNumberOfPtes : 0x1e8b00

 

The MmSt allocations contain these PTEs so all we need to do is multiply this by the size of a PTE to get the total size of the MmSt allocations for this control area. Note that there may be multiple allocations for this control area, but this number will reflect the total size all these allocations.

 

5: kd> ?? 0x1e8b00 * sizeof(nt!_MMPTE)

unsigned int 0xf45800

 

So now we know the MmSt size in bytes for a single control area, or mapped file. What if we would like to see the totals for all mapped files from the !memusage output? First, place the !memusage output in a text file and remove all header information. You will also need to remove all tail information including the page file and process summaries. Every line should look like these.

 

8c62a638 1108 945868 3064 0 0 0 mapped_file( $Mft )

8b06ac18 516 0 0 0 0 0 No Name for File

 

We want to include the “No Name for File” entries since those are valid mapped files even though the name could not be located. Next strip out everything but the control area address. You can use Excel or any other tool that allows you to select and delete columns in a text file. Now we have a file with a single column of all the control areas on the system. The following debugger command script can be used to process this file.

 

$$ countptes.txt script

r $t2 = 0;

$$ Replace the memusage.txt file name with your file name.

.foreach /f (ca "memusage.txt") {

    r $t1 = @@c++(((nt!_CONTROL_AREA *)(0x${ca}))->Segment->TotalNumberOfPtes);

    .printf "Control Area %p : %d\n", ${ca}, @$t1;

    r $t2 = @$t2 + @$t1;

}

.printf "Total PTEs : %d\n", @$t2;

.printf "MmSt size : %d bytes\n", (@$t2 * @@c++(sizeof(nt!_MMPTE)));

 

The following command will execute the script.

 

5: kd> $$><countptes.txt

 

This will show the number of PTEs for each control area, followed by a summary.

 

Total PTEs : 62790244

MmSt size : 502321952 bytes

 

A common high user of MmSt allocations is $Mft. The cache manager will hold the MmSt allocations for these file system metadata files at a cost of up to 4 files per PTE. This technique can be used to determine how much $Mft is using MmSt pool memory by first using findstr at a command prompt to isolate just those values from the !memusage output.

 

C:\Projects>findstr /c:"$Mft" memusage.txt >mftusage.txt

 

After stripping out the control area addresses with Excel and running the command script you’ll have the size of the MmSt allocations for just the $Mft files. If this is consuming most of the MmSt bytes then you are limited to the options mentioned at the beginning of this article. There may be other options if something else is the primary user but it will likely involve reducing some heavy load on the system.

 

-Bryan

Comments

  • Anonymous
    May 13, 2008
    This is an awesome explanation.

  • Anonymous
    July 07, 2008
    The comment has been removed

  • Anonymous
    September 08, 2008
    Hi, The mmst tag is related to the pool paged memory, and PTE is related to nonpaged pool memory, if mmst tag is consuming lot of memory then the paged pool is not recycled, this could be most common in windows 2000, all you have to do is trim the memory by making an entry in the registry you can make it at 60%, so that after reaching the maximum memory the paged pool will be recycled

  • Anonymous
    January 03, 2014
    Thank you for sharing your knowledge. I've slightly modified your script to work with a Windows 2008 R2 64bit dump I was working with, and to accept address file as a script parameter. $$ countptes.txt script usage: $$  $$>a<"C:Path to ScriptCountPTEs.txt" "C:some foldersome address file.txt" .echo Using ${$arg1} as input file. r $t2 = 0; .foreach /f (ca "${$arg1}") {    r $t1 = @@c++(((nt!_CONTROL_AREA *)(0x${ca}))->Segment->TotalNumberOfPtes);    .printf "Control Area %p : %I64dn", ${ca}, @$t1;    r $t2 = @$t2 + @$t1; } .printf "Total PTEs : %I64dn", @$t2; .printf "MmSt size  : %I64d bytesn", (@$t2 * @@c++(sizeof(ntkrnlmp!_MMPTE)));

  • Anonymous
    February 06, 2014
    @Olegas use xperf today: channel9.msdn.com/.../Defrag-Tools-48-WPT-Memory-Analysis-Pool [Xperf use is limited to relatively small timeframes, so it is not applicable to all scenarios.  We have an article describing it's use for pool leaks at http://blogs.msdn.com/b/ntdebugging/archive/2012/11/30/troubleshooting-pool-leaks-part-7-windows-performance-toolkit.aspx .]

  • Anonymous
    March 08, 2014
    I see roughly eighty thousand mapped_file s open from the dump, they all are oldest log files that we have on disk just for retention sake. Nobody should be opening them for any purpose, is there a way to track this further down to a process opening them, might backup/AV..etc? Could I request an expert advice to pinpoint this to a process?  Thanks in advance. [Tools such as Process Explorer can be used to determine what processes have handles open to the files.]