Symptoms: Lack of Free System Page Table Entries (PTEs) and system-wide delays (I/O request failures)
Introduction
The purpose of this article is to provide prescriptive guidance on how to troubleshoot free system page table entries (PTEs) in regards to Windows performance analysis.
Start with the following performance counters to analyze free system PTE’s:
· \Memory\Free System Page Table Entries
A page table is the data structure used by the Windows Virtual Memory Manager (VMM) to store the mapping between virtual addresses and physical addresses in memory. The performance counter Free System Page Table Entries is the number of page table entries not currently used by the system.
From the process perspective, each element of virtual address conceptually refers to a byte of physical memory. It is the responsibility of the Virtual Memory Manager (VMM) in conjunction with processor memory management unit (MMU) to translate or map each virtual address into a corresponding physical address.
The VMM performs the mapping by dividing the RAM into fixed-size page frames, creating system PTEs to store information about these page frames, and mapping them. System PTEs are small kernel-mode buffers of memory that are used to communicate with the disk I/O subsystem and the network. Each PTE represents a page frame and contains information necessary for the VMM to locate a page.
Note: Troubleshooting System PTE’s is explained in more detail at in the “Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues” article mentioned in the References section below.
This article is grouped by symptoms, then by possible causes.
Symptoms: Lack of Free System Page Table Entries (PTEs) and system-wide delays (I/O request failures)
Applies to:
- 32-bit Windows Server 2003 (all editions) unless otherwise specified
- 32-bit Windows XP (all editions) unless otherwise specified
- 32-bit Windows Server 2000 (all editions) unless otherwise specified
Symptom Details:
· Lack of Free System Page Table Entries (PTEs): Use the “Memory\Free System Page Table Entries” performance counter for values under 5000. Alternatively, the !pte command in the kernel debugger can be used to examine PTEs.
- Periodic system-wide delays: System-wide delays or hangs that occur regularly or occur during elevated load on the system. This is measured by the user experience and I/O response times of the system.
Possible Cause: Use of Physical Address Extension (PAE) Kernel
How to Diagnose
· To determine if the Windows 2003 server is booted with the PAE kernel, by checking for a value of 1 in the registry key, “HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PhysicalAddressExtension”. Alternatively, the boot.ini file can be searched for the PAE boot switch.
Note: when execution protection is enabled on 32-bit Windows, the system automatically boots in PAE mode (automatically selecting the PAE kernel).
· The PAE switch allows the Windows operating system to address more than 4GB’s of physical memory and in certain environments the PAE switch is automatically enabled. On x86 systems, the page table index is 10 bits wide (9 on PAE), allowing you to reference up to 1024 4-byte PTEs. Therefore, the /PAE switch causes PTE entries to use twice the normal allotted virtual address space. For more information, see “Microsoft Windows Internals” by Mark E. Russinovich and David Soloman.
Possible Solutions and/or Recommendations
· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
· Remove the /PAE switch: If not needed, then remove the /PAE switch from the boot.ini file.
Possible Cause: Use of the /3GB switch
How to Diagnose
· The /3GB switch significantly reduces the number of free system PTE’s. To check if the /3GB switch is enabled use one of the following methods:
o Open the boot.ini file and look for the /3GB switch on the most recently used boot option.
o Check the value of the registry key: “HKLM\SYSTEM\CurrentControlSet\Control\SystemStartOptions”
o Check the value of the WMI property root\cimv2\Win32_OperatingSystem.MaxProcessMemorySize for a value close to 3,145,536 (3GBs) and greater than 2,097,024 (2GBs).
· Refer to the chart below for the starting amount of free PTE’s in different operating system configurations. Note the different in the amount of free PTE’s when the /3GB switch is enabled.
Possible Solutions and/or Recommendations
· Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” https://support.microsoft.com/kb/316739.
Note: For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch. For more information, go to https://technet2.microsoft.com/windowsserver/en/library/c5ccbaec-f552-4f61-a488-8ee3330d1eeb1033.mspx.
· Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.
· Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.
· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
Possible Cause: Too much physical memory
How to Diagnose
· Refer to the “Estimated Kernel Resources Chart” in the More Information section for the starting amount of free PTE’s in different operating system configurations. Note the how increasing the amount of physical memory on the server reduces the number of free PTE’s on the server.
Possible Solutions and/or Recommendations
· Reduce the amount of physical memory: Reduce the amount of physical memory to free up more virtual memory for system PTE’s.
· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
Possible Cause: High resource consumption and/or poorly written device drivers
How to Diagnose
Possible Solutions and/or Recommendations
· Remove any unnecessary third-party drivers and drivers not listed on the Hardware Compatibility List (HCL): Drivers that are not on the Hardware Compatibility List (HCL) have a higher likelihood of causing system problems and/or misuse of system resources. For more information about the Microsoft Hardware Compatibility List (HCL) go to, https://winqual.microsoft.com/hcl/.
· Use /BASEVIDEO: Use the /BASEVIDEO boot switch in the boot.ini file or a generic video driver to free up system page table entries. Video boards use the system page table entries to map their buffers in kernel space. This usage competes with the need for system page table entries by Microsoft Exchange.
· Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” https://support.microsoft.com/kb/316739. For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch.
· Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.
· Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.
· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, go to https://msdn2.microsoft.com/en-us/library/bb870880(VS.85).aspx.
Possible Cause: High number of mailboxes on Microsoft Exchange Server
How to Diagnose
Possible Solutions and/or Recommendations
· Reduce the number of mailboxes on the server and/or remove the public folder role from the mailbox server.
High number of Microsoft Exchange mailbox stores that have the public folder server set as their default public store.
· Reduce the number of mailbox stores that have the public folder server set as their default public store. This action will reduce the number of clients (and, as a result, the number of user sessions) on the public folder server.
Estimated Kernel Resources Chart
This chart shows the estimated, initial maximum size of kernel resources on 32-bit Windows Server 2003 computers when using PAE, /3GB, with booted different physical memory sizes. Use the “!vm” command when doing a kernel debug to find the actual kernel resources.
Memory |
Default ( /PAE for 6-16GB ) |
/3GB |
1GB |
Free System PTE: 51k Paged Pool: 282MB Non Paged Pool: 212MB |
Free System PTE: 32k Paged Pool: 163MB Non Paged Pool: 131MB |
2GB |
Free System PTE: 196k Paged Pool: 360MB Non Paged Pool: 262MB |
Free System PTE: 16k Paged Pool: 262MB Non Paged Pool: 131MB |
3GB |
Free System PTE: 195k Paged Pool: 360MB Non Paged Pool: 262MB |
Free System PTE: 14k Paged Pool: 262MB Non Paged Pool: 131MB |
4GB |
Free System PTE: 106k Paged Pool: 336MB Non Paged Pool: 285MB |
Free System PTE: 15k Paged Pool: 258MB Non Paged Pool: 154MB |
6GB |
Free System PTE: 186k Paged Pool: 366MB Non Paged Pool: 262MB |
Free System PTE: 12k Paged Pool: 239MB Non Paged Pool: 131MB |
8GB |
Free System PTE: 182k Paged Pool: 366MB Non Paged Pool: 262MB |
Free System PTE: 12k Paged Pool: 225MB Non Paged Pool: 131MB |
12GB |
Free System PTE: 175k Paged Pool: 366MB Non Paged Pool: 262MB |
Free System PTE: 12k Paged Pool: 196MB Non Paged Pool: 131MB |
16GB |
Free System PTE: 167k Paged Pool: 366MB Non Paged Pool: 262MB |
Free System PTE: 12k Paged Pool: 169MB Non Paged Pool: 131MB |
References
- Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues
https://www.microsoft.com/downloads/details.aspx?FamilyID=ed0e8084-abf7-4c00-ba6a-7d658cdb052a&DisplayLang=en - “Windows Internals Microsoft Windows Server 2003, Windows XP, and Windows 2000” by Mark E. Russinovich and David Soloman.
- Comparison of 32-bit and 64-bit memory architecture for 64-bit editions of Windows XP and Windows Server 2003
https://support.microsoft.com/kb/294418 - System Pages
https://technet2.microsoft.com/windowsserver/en/library/c5ccbaec-f552-4f61-a488-8ee3330d1eeb1033.mspx?mfr=true - How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB
https://support.microsoft.com/kb/316739
Contributors:
Clint Huffman, Shane Creamer, Rick Anderson, Maximilian Silva, Matthew Walker, Pavel Lebedynskiy, John Rodriguez, Mike Lagase, Yong Rhee
Comments
Anonymous
January 01, 2003
PingBack from http://liludrivers.net/2008/04/08/symptoms-lack-of-free-system-page-table-entries-ptes-and-system/Anonymous
January 01, 2003
It was a dark and stormy night… actually it was sunny and clear, but just thought I’d start this blogAnonymous
June 23, 2009
Hello.........This article is just wonderful and gives detail steps about how to go troubleshooting this problem of low on system page table entries. Thanks a lot for this wonderful article. Gives simple and awesome explanation.Anonymous
June 23, 2009
The comment has been removedAnonymous
November 24, 2009
The comment has been removed