Who ate my memory?
The consumer versions of 32-bit Windows XP and Vista have a stated limit of 4 GB RAM, but a practical limit of about 3.1 GB. A lot of partial explanations have been floating around, so I thought I would try my hand at clearing up the issue. (Wish me luck!)
The design of the Intel 386 architecture supported access to up to 4 GB of physical memory (32-bit physical addresses) and unlimited virtual memory (4 GB at a time via 32-bit virtual addresses). 4 GB of physical memory seemed quite unthinkable at the time the chip was released, so the actual CPU did not have enough address pins to actually do this. Back then a 32-bit address space seemed extravagant for anything less than a supercomputer or mainframe. Nowadays, you can get 4 GB for under $400, and what was unthinkable in 1986 is within reach of anybody thinking about a new computer.
So at least I can access 4 GB, right? Nope.
The original IBM PC’s processor could access 1024 KB of physical address space, but you could only use 640 KB for RAM. The remaining 384 KB of address space was reserved for memory-mapped hardware and ROM. A similar situation exists with current systems: hardware reserves large chunks of the upper 1 GB of physical address space. Because of these reserved areas, a system with a 32-bit physical address space will be limited to somewhere around 3.1-3.5 GB of RAM.
To overcome the 32-bit limitation, recent x86 CPUs (Pentium Pro and later) have 36 address pins and can address 64 GB of RAM. The original design of the x86 32-bit protected mode only provided access to 32-bit addresses, so PAE (Physical Address Extensions) mode was created to allow access to 36-bit addresses.
PAE mode changes the layout of the page tables. Page tables map virtual addresses to physical addresses. Without PAE, the 32-bit virtual addresses map through 2 levels of page tables (1 level for huge pages) and are translated to 32-bit physical addresses. With PAE, the 32-bit virtual addresses map through 3 levels of page tables (2 levels for huge pages) and are translated to 64-bit physical addresses.
PAE doesn’t do anything to the virtual memory limit. Pointers are still 32 bits, so a process can only access 4 GB of address space at a time. However, using PAE, two or more processes could each access a different 4 GB of physical memory. With proper operating system support (i.e. AWE on Windows operating systems) PAE also allows a process to allocate additional memory outside its normal address space, then swap portions of that additional memory into its address space as needed.
So PAE is the answer, right? Well, maybe…
One thing that can prevent access to more than 4 GB of RAM is motherboard design. PAE can only access 64 GB of memory if all 36 address pins are properly wired up on the motherboard. This is not always the case, since those extra 4 wires make the motherboard just a little bit more expensive to design and manufacture (and use just a little bit more power). Many motherboards (especially on laptops) only have 32 address pins connected. If that is the case, no OS will be able to access more than 4 GB of address space.
Another hardware limitation is the ability of the chipset to remap RAM. If you have 4 GB of RAM, and 600 MB of address space is used up by PCI/AGP reserved areas, the only way to access the top 600 MB of RAM is to remap it into the addresses above the 4 GB boundary. Not all chipsets are able to do this, so some systems will just waste any RAM that happens to be shadowed by a PCI/AGP reserved region.
My BIOS reports 4 (or more) GB of RAM, I’ve enabled PAE, and I still only see 3.1 GB. What gives?
Unless you’re running one of the advanced server varieties of Windows, you won’t see more than 4 GB of physical memory. This is a limitation of Windows designed (I assume) to encourage people building expensive servers to pay more for Windows than those who are using it for normal day-to-day activities.
As for that last 0.9 GB, it all comes down to drivers and system stability. Not all drivers behave well in the presence of 64 bit physical addresses. Many driver authors assume that only the bottom 32 bits of the physical address are valid. Others don't properly handle the creation of bounce buffers when necessary (they’re needed when transferring data from a hardware device to/from a buffer that is above the 4 GB mark in physical memory).
Windows XP originally supported a full 4 GB of RAM. You would be limited to 3.1-3.5 GB without PAE, but if you enabled PAE on a 4 GB system with proper chipset and motherboard support, you would have access to the full 4 GB. As more people began to take advantage of this feature using commodity (read: cheapest product with the features I want) hardware, Microsoft noticed a new source of crashes and blue screens. These were traced to drivers failing to correctly handle 64-bit physical addresses. A decision was made to improve system stability at a cost of possibly wasting memory. XP SP2 introduced a change such that only the bottom 32 bits of physical memory will ever be used, even if that means some memory will not be used. (This is also the case with 32-bit editions of Vista.) While this is annoying to those who want that little bit of extra oomph, and while I would have liked a way to re-enable the memory “at my own risk”, this is probably the right decision for 99.9% of the general population of Windows users (and probably saves Dell millions in support costs). See the relevant KB article and a TechNet article for details.
Some of the server Operating Systems still allow the use of larger amounts of memory. I’m guessing that this is done with the assumption that higher quality parts will be used and drivers will be more likely to have been tested in PAE mode with large addresses.
Side-note: PAE is also related to page execution protection, called "hardware DEP" (Microsoft term), "NX" (AMD term), and "XD" (Intel term). In 32-bit x86 processors, this can only be used in PAE mode. This is why you might see PAE mode used even on systems with less than 4 GB of memory.
Performance note: 3-level page table lookups are inherently slower than 2-level page table lookups. However, the processor has substantial dedicated circuitry that usually eliminates most of the performance impact.
Comments
Anonymous
March 29, 2007
The comment has been removedAnonymous
March 29, 2007
I'm not sure what this has to do with the EULA. I haven't looked at the source code, but from the KB articles I read, it seems like the 32-bit physical address limitation is built into the kernel. I don't think there is any kind of registry setting that would remove the limitation. I could be wrong. This isn't about RAM quality. This is about device driver quality. In many cases, the difference between cheap hardware and great hardware is not the hardware -- it's the quality and stability of the driver. Unfortunately, it is very difficult to determine whether a driver is well-written. WHQL is a good start.Anonymous
April 17, 2007
But how come, that Vista only shows 2GB of RAM, even with PAE enabled? BIOS reports 4GB, so VIST should say something like 3.1GB, right? Not 2.0GB.Anonymous
May 11, 2007
I also have DEP, and PAE as a result, enabled, and have 4 GB of ram that is detected correctly on my 64-bit system, but on my 32-bit it is not.Anonymous
May 11, 2007
Sorry, I meant to specify that I am only reading 2GB of ram in my 32-bit VISTA, not in the vacinity of 3 as others have done successfully.Anonymous
May 13, 2007
Nelson: That's weird. This sounds like a motherboard limitation or a motherboard driver issue, but I can only guess.Anonymous
May 21, 2007
Thanks for detailed information. But my PC has Intel 3.0 GH 4 Processor 631, with 1.5GB RAM. The physical memory shown properly in BIOS but in System properties of XP SP2, it shows 1.37GB RAM. Is it required to enable PAE here? I guess as its not more than 4GB, PAE setting is not required. Where is another .13G RAM? Regards, PavanAnonymous
June 10, 2007
Shame on the driver writers that still write drivers that does not work with PAE!!! PAE support existed since Windows 2000!!!!!Anonymous
June 18, 2007
I have read the above, and appreciate those who try to shed light on this situation. I have: ASUS P5B MB 4 SDRAM 1 GB DIMMS Intel socket 775 processor (6400 @ 2.13 GHz) Windows Server 2003 Enterprise Edition with PAE enabled - and the /SOS switch in the BOOT.INI file MB shows 4096 GB upon boot Windows shows 4096 GB when booting with /sos switch Windows shows 4096 GB in properties BUT - Microsoft Virtual PC 2007 and Microsoft Virtual Server and other 'utilities' only show 2048 GB of available memory within the application(s) - so even though system shows me 4096 GB I'm thinking really only 2 GB is available for use because of PAE being enabled. If I turn off PAE then I have a bit over 3 GB available. What kind of a twist is this throwing into the explanations above, if any? Steven Shipee slshippee@comcast.netAnonymous
June 20, 2007
To Pavan: You are not even close to the amount of memory that would require PAE. Check to see if your system is using an on-board video chipset instead of a dedicated video card. Usually when you see a chunk of memory missing in Windows like that, it's because the system has reserved that memory for video memory, or something like that. A quick calculation shows that your are missing about 128MB of memory, so that would make sense. Your BIOS should have a setting for how much memory is reserved for video.Anonymous
July 24, 2007
The comment has been removedAnonymous
August 14, 2007
The comment has been removedAnonymous
August 24, 2007
This is my issue: 4 Physical gig memory in this system. Running windows server 2003/SQL. Memory shown as 2.91 GB. Bios shown 3 BG total. When I updated bios to the latest still unable to get it to see 4BG in bios. Not I've tried 4 different motherboards same thing. So I've done ini file, pae,bios, not sure what's going. Any suggestion? ThanksAnonymous
September 14, 2007
I believe 64bit Windows Vista fixes this problem so I think the solution is to FINALLY move beyond 32-bit everything. Driver manufactures and application developers need to focus on 64bit support Microsoft needs to encourage this through the removal of 32bit support on a determined schedule. I believe they have already taken the first step towards doing this by only releasing Exchange 2007 in 64bit. Excellent! Do I need more than 3GB of memory for Windows XP/Vista 32bit?... maybe it would be nice but I think I'll move to 64bit and into the future and start taking my lumps now. My 2 cents. Adam www.professionalintegrations.comAnonymous
October 10, 2007
I lately got my hands on 4GB of RAM which I tried to put to use in my Laptop. My Fujitsu Siemens Celsius H240 is a Intel CoreDuo based Platform and I had hoped that I would be able to acess all of the memory. Long story short: the circuit / BIOS combinatAnonymous
October 31, 2007
Hi i got the same problem here too i got 4gb RAm my bios confirms it but my Vista only uses 2 GB. any idea what i can do about it??Anonymous
October 31, 2007
The comment has been removedAnonymous
November 01, 2007
The comment has been removedAnonymous
November 01, 2007
Silvereagel - I really don't know. I have two video cards on my machine, so I can only access 3 GB. When I remove one of them, I can access 3.5 GB. It depends on the hardware you have installed. But I don't know the exact reasons why it doesn't work for you. Maybe your BIOS is configured to remap the memory and it is remapping too much of it. Normally the BIOS will only remap the part that conflicts with hardware. But Vista-32 can't use any of the remapped memory (since it is remapped above the 4GB mark).Anonymous
December 22, 2007
No, si usas un sistema operativo de 32 bits no puedes usar más de 4 GiB de memoria RAM. Punto. Con 33 bits sólo puedes acceder a 2 elevado a 32 posibles direcciones. Existen mecanismos propuestos para tratar de acceder...Anonymous
December 22, 2007
Running Linux eliminates these problems.Anonymous
December 22, 2007
Not true - Linux can't re-wire your motherboard. Linux actually faces many of these same problems - the only difference being that Linux is happy to let you shoot yourself in the foot with driver quality issues with 64-bit addresses, where Windows will try to protect you. But this is only applicable if you motherboard and your chipset can support PAE, which a very large number can not.Anonymous
December 22, 2007
this article was referenced on slashdot on 22-DEC-2007: http://hardware.slashdot.org/article.pl?sid=07/12/22/076211Anonymous
December 22, 2007
"Linux is happy to let you shoot yourself in the foot with driver quality issues with 64-bit addresses, where Windows will try to protect you." On Linux, drivers are free software. If they have bugs, we fix them. We don't have to add workarounds to "protect" the user from 3rd party propietary trash.Anonymous
December 23, 2007
All you people missing 128 MB or 64 MB of RAM probably have on-board video. Intel and Pavan, this isn't an issue of the OS not accessing that RAM -- it's just your style of MB.Anonymous
January 01, 2008
http://yuhong386.spaces.live.com/blog/cns!57E2793D0C53276F!147.entryAnonymous
January 21, 2008
"On Linux, drivers are free software. If they have bugs, we fix them. We don't have to add workarounds to "protect" the user from 3rd party propietary trash." I agree with open-source development, however the word "protect" isn't evil. Protection is good. The facebooking, myspacing, e-mailing, pornsurfing end user shouldn't need to be aware of hardware configuration. 1 bug on MS software = media fieldday 100 bugs on Linux = another day at the office.Anonymous
January 25, 2008
So this problem only arises, if using a 32bit architecture? or is this also a problem for 64bit architectures? I'm thinking of buying a new PC and wanted to have 4GB RAM in it (and be able to use this 4 GB).Anonymous
January 30, 2008
Do Mac (is this a dirty word?) users have these problems?Anonymous
February 03, 2008
I have a 64 bit prosseser and I'm looking fowerd to getting vista ultamate 64 bit.Anonymous
February 04, 2008
Before I try this I thought I would ask a couple of questions (before we start...I am NO guru when it comes to computers, so bear with me please).
- I run Vista64 with a total of 8GB, shows 3.83GB. Total VM of 9.58Gb, shows 5.94GB.
- I'm trying to run Autodesk Inventor (32) and have a large assembly that "dumps" in the (roughly) 4GB area. I imagine I'm hitting the 4GB "limit".
- I've seen where adding /PAE will get me the full 4GB, but what about the other 4GB above that. Will I ever be able to access the full 8GB with Inventor? Sorry for any "stupidity" on this, but I'm on a "non-supported" machine at the company I work for...so I'm the IT guy for this (and yes, we're in trouble). Thanks
Anonymous
February 05, 2008
Mac users are shielded from these problems by not having infinitely varying hardware and software configurations. On hardware that doesn't allow more than 4 GB of address space, Apple limits the RAM to a lower value, generally 2 GB. Higher-end machines with higher RAM capacity are designed with all the necessary hardware support included. The Mac OS X kernel and Unix layer have been 64-bit for a while, and Leopard (OS X 10.5) also offers full support for 64-bit applications with graphical user interfaces.Anonymous
February 06, 2008
I am running Windows Vista x64 and I installed 4GB of RAM, the BIOS shows 4096MB RAM but Windows Vista shows 3070MB RAM. This is seriously f*cked up.Anonymous
February 11, 2008
The comment has been removedAnonymous
February 11, 2008
Gabriel: I see exactly the same problem. I have just upgraded my laptop - now I have 2 x 2GB RAM. In BIOS I see 4GB, but on Vista 64 only 3GB.Anonymous
February 13, 2008
Windows xp/vista 32bit can only use 2gb of memory unless you enable the /3gb in the boot.ini. It cannot utilize the memory. you want 4gb or more of ram you need to upgrade to 64bit windows.Anonymous
February 13, 2008
The comment has been removedAnonymous
February 14, 2008
yeah my PC with 2 OS (Vista 32-bit and XP sp2) just detected 3,5GB PC 800 DDR2 but in my Bios detected 4 GB.. how to fix it .. maybe vista 64-bitAnonymous
February 15, 2008
Yes, anything 64-bit It will read 3.5gb ram but it will still only use 2gb for the user regardless. unless you do the 3gb in the bootini but even taht can cause problems. 64bit is the way to go for up to 8gb of ram utilizationAnonymous
February 20, 2008
The comment has been removedAnonymous
March 01, 2008
Gabriel/Andrew/Nick: If you have a 64-bit edition of Windows but you still only get 3.5 GB of RAM, it is a motherboard or BIOS limitation (the motherboard is not remapping the memory). In some cases, this something that can be enabled in the BIOS options, but on laptops it is often simply a solid limit. Hug: Actually, the /3GB switch is much more complicated than that and has NO EFFECT on how much RAM can be used by Windows. Normally on a 32-bit operating system, Windows reserves 2 GB of address space for itself and gives each process 2 GB of address space. With the /3GB switch, Windows limits itself to 1 GB of address space and processes can use 3 GB of address space if they are specially marked as "large address aware". Note that this refers to address space, not RAM. Also note that this switch has a significant impact on many areas of the system (maximum amount of disk cache, maximum number of threads, windows, processes, TCP/IP sockets, etc.) since there is less space available for the operating system, so it should only be used to correct a specific problem where a memory-intensive program (SQL Server or Exchange) runs out of address space. I personally kind of like Vista, but there are definitely pros and cons. It is certainly more secure and a lot of things just work better, but it also uses more memory and can be a bit annoying. So I can understand your opinion.Anonymous
March 03, 2008
Dear All, I have Installed 8GB of RAM (2GB each 4 Pcs) in windows XP Pro 32 bit, but it is showing only 3.5 GB. how can i enable the full 8 GB? please give me the simple way to do.... send me to my email please.... ashruak@hotmail.com ashruak@gmail.com ashrafa@petrominoils.com thank you in advance AshrafAnonymous
March 03, 2008
Dear All, I have Installed 8GB of RAM (2GB each 4 Pcs) in windows XP Pro 32 bit, but it is showing only 3.5 GB. how can i enable the full 8 GB? please give me the simple way to do.... send me to my email please.... ashruak@hotmail.com ashruak@gmail.com ashrafa@petrominoils.com thank you in advance AshrafAnonymous
March 23, 2008
There have been a number of comments regarding os x 10.5. In fact, os x leopard kernel is 32-bit, NOT 64-bit. Se for example: http://developer.apple.com/documentation/Darwin/Conceptual/64bitPorting/transition/chapter_3_section_4.htmlAnonymous
March 28, 2008
Okay I also have this problem.. Been running 2x1 gigs of ram for some time, and now upgraded with further 2x1 gigs.. BIOS detects 4 gigs but windows (xp pro) only 2.. Been playing a bit with the /PAE and /3gb switches with no result and can now understand that these are effectless in my case. dcook, you were explaining something about the /3gb switch, something about letting some programs use larger amounts of ram or adress space.. How to do this? Think that would solve my problem, as my music software demands at least 2 gigs of FREE memory!Anonymous
May 16, 2008
Re J's comment on Linux: Just for anyone coming across this page for info, J's comment re Linux having this problem too is misleading. Of course if the hardware cannot support the memory addressing extension, no 32-bit OS can. So obviously there is a mobo and chipset prerequisite. Having said that, since the 2.6 kernel, Linux 32-bit supports up the 64GB of addressable memory. It is not necessary to use the 64-bit kernel, and there are no special driver requirements or issues using the 32-bit drivers. Therefore, the remark re 64-bit address driver issues (which I assume back references the problem described with Windows drivers) is not only not applicable, it is uninformed (to put it kindly). The short of it is that this limitation is a MS design decision. Specific to the desktop. Period. Don't like it? Go 64-bit or go Windows Server or go Mac or go Linux.Anonymous
June 02, 2008
I'm running 2x2GB for a total of 4GB with Windows XP Pro SP2 32 BIT. Only showing 3.2GB of ram in windows, but shows the full 4GB in bios. I would have gone with 64 bit but the main software I use for my business doesn't support 64bit. BLAH! Intel Q6600 Quad 4GB or XMS Ram (showing 3.2gb) 2.5 TB hard drive space. 1 TB RAID NVidia Quadro 1500 FXAnonymous
June 23, 2008
Hi, I boot using Win PE.(64 Bit architecture) I have 64 Gb Ram but task manager shows 2GB of Ram only. Any proper explanation for this will be greatly appreciated.
- Sam
Anonymous
August 26, 2008
AppleInsider has a good rundown on how Mac OS X and Windows compare (and a historical overview of memory addressing) in "Road to Mac OS X 10.6 Snow Leopard: 64-Bits" at: http://www.appleinsider.com/articles/08/08/26/road_to_mac_os_x_10_6_snow_leopard_64_bits.htmlAnonymous
September 02, 2008
The comment has been removedAnonymous
September 19, 2008
The comment has been removedAnonymous
October 12, 2008
Half way through that article you decided to say 64-bit addresses instead of 36-bit. I found that amusing although not distracting from the quality of the content.Anonymous
January 05, 2009
The comment has been removedAnonymous
February 02, 2009
The comment has been removedAnonymous
February 20, 2009
Yours is one of the best pages I have found on this subject. Indeed, it's one of the few that are commendable. But I have to ask why don't you point out that the means to do: while I would have liked a way to re-enable the memory “at my own risk”, this is probably the right decision for 99.9% of the general population of Windows users exists anyway, through boot options? It's not as if Microsoft would have had to do extra work for what you say you want. All that Microsoft would have needed to do is set pae=ForceDisable and truncatememory=0x100000000 as defaults at installation. Manufacturers who were confident of their hardware and drivers could remove these when setting up. Your 99.9% would be protected. Instead, 32-bit Vista retains the code for accessing memory above 4GB but the licensing mechanism is used so that memory above 4GB is discarded. If Microsoft were more open about this being the way the limit is imposed, then the limit might seem less disagreeable. At least it could then be discussed on the basis of what actually is done, and those driver and hardware issues could be assessed independently. Better yet, 64-bit Windows could be assessed on its merits as an operating system for the mass market, instead of being promoted misleadingly as being necessary for using 4GB and more of RAM. Geoff.Anonymous
March 12, 2009
The comment has been removedAnonymous
July 01, 2009
free music tv - yabancı müzik dinleAnonymous
September 11, 2009
I am running Windows Vista x64 and I installed 4GB of RAM, the BIOS shows 4096MB RAM but Windows Vista shows 3070MB RAM. This is seriously f*cked up.Anonymous
November 29, 2009
Hahaha! It's funny how every second comment seems to be the reincarnation of either: "BIOS shows 4 GB or RAM while Windows only shows 3.5 GB !!!!1 Halp me pl0x !!!1one!" or "I 'upgraded' my PC/Laptop/Toaster to 1 Bazillion GB or RAM but it just won't use more than 2/3/4 GB !!i!1!eleven!" Did you people even read the article? Sorry but if you keep posting the same questions over and over again, even though they were answered a hundred times already, then you are nothing but ignorant fools who shouldn't be using (or at least building/upgrading) computers. You are simply not smart enough for that task, sorry.Anonymous
December 23, 2009
Turkiyenin en kaliteli sesli chat sitesi.Anonymous
December 23, 2009
Turkiyenin en kaliteli motosikletr sitesi.Anonymous
December 23, 2009
Turkiyenin en kaliteli ruya tabir sitesi.Anonymous
December 23, 2009
Turkiyenin en kaliteli kameralı chat sitesi.Anonymous
September 03, 2010
Knowing this is an old thread, I'll risk having no answer... I understand that the OS Kernel/hardware/whatever will reserve space on my memory that will not be available to the rest of the applications, that is why I have 4GB or RAM but it shows only 3. Now the question I have is this: when my computer had only 2GB RAM (and we're talking about the same computer), Windows showed me the whole 2GB as available. Why did it not show me only 1GB (as I believe it should have because of the reserved space)?