FSX and Win32 Process Address Space

I see multiple threads on /3G enabling FSX to get around an “Out of Memory” error that some flyers see when they max out the settings. So let’s talk about what this all means.

Normal Win32 processes have 2G of address space, out of the 4G available. The remaining 2G is left to the OS for its needs, drivers, etc. This is completely independent of physical RAM - this is how much virtual address space each process gets.

Here is the definitive discussion, on MSDN, about process address space. The article states:

The virtual address space for 32-bit Windows is typically divided into partitions as follows.

Range	Usage
Low 2GB (0x00000000 through 0x7FFFFFFF)	Available to the process.
High 2GB (0x80000000 through 0xFFFFFFFF)	Reserved for the system.

 

So you can see there is limited usage of the high order bit, normally. Note the links to “4GT Ram Tuning” and “IMAGE_FILE_LARGE_ADDRESS_AWARE”.

In the case of 4GT Ram Tuning, the article states:

If 4GT RAM Tuning is enabled, the virtual address space for 32-bit Windows is divided into partitions as follows.

Range	Usage
Low 3GB (0x00000000 through 0xBFFFFFFF)	Available to the process.
High 1GB (0xC0000000 through 0xFFFFFFFF)	Reserved for the system.

 

And

Only processes that have the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in the image header have access to memory above 2 gigabytes (GB).

The process address space can be adjusted between 2 GB and 3 GB by using the /USERVA switch in boot.ini. The system address space adjusts as needed.

This is telling us 2 things:

1. there is a way to expand the process address space for Win32 processes

2. there is a way to mark applications to make use of this expanded address space

And you need both of these actions performed for this to actually show benefits, e.g. you need to both enable 4GT Ram Tuning for the OS and you need to mark the application.

This thread on avsim.com contains instructions on how to do this for the OS and to FSX for yourself. Previously, I was recommending against this because there was a known bug in how D3DX handles effects when this flag was set. And this bug can cause crashes when D3DX effects use memory above the 2G line. There is a clue about this in the “4GT Ram Tuning” article on MSDN, where the article states:

Use the following guidelines to enable 4GT support in applications:

· For a DLL that loads near the 2-GB boundary, there is a region of the 2-GB space where contiguous memory cannot be allocated by using the VirtualAlloc function.

· To retrieve the amount of total user virtual space, use the GlobalMemoryStatus function. Always detect the real value at runtime, and avoid using hard-wired constant definitions such as: #define HIGHEST_USER_ADDRESS 0xC0000000.

· Avoid signed comparisons with pointers, because they might cause applications to crash on a 4GT-enabled system. A condition such as the following is false for a pointer that is above 2-GB: if (pointer > 40000000).

· Code that uses the highest bit to tag items (that is, data value versus an address value) fail. For example, a 32-bit word might be considered a user-mode address if it is below 0x80000000, and an error code if above. This is not true with 4GT.

 

D3DX was running afoul of one of these rules ( HINT: one of the last 2 rules ) and this could cause a crash when using virtual address space above 2G was enabled on FSX. While I still don’t recommend that users play with things they don’t understand, word we hear is that the June 2007 DirectX redist addresses the issue in D3DX. So one element of risk is gone.

This switch is supported on both Vista and XP Professional, although I consistently hear reports of the “Out of Memory” issue on Vista and not on XP.

Note the article also states:

Setting this flag, and then running the application on a system that does not have 4GT support should not affect the application.

So it should be completely safe for Aces to do this, and only flyers who had tweaked their OS would see this benefit. So indeed we have planned an investigation about this. That is not a commitment to use this switch in a future release - that is a mention to the community that we are investigating.

Note this switch needs to be used with care, since PCI-Express maps the entire address space of the graphics card into the OS address space. If you set the app to use 3G with a 768m graphics card, that leaves 256m for the OS. A better setting is 2560, eg 2.5G, since that leaves 768m for the OS with a 786m graphics card. Much more reasonable.

So how is this helping with the “Out of Memory” error? In the "Out of Memory" case, 2 things can be happening to cause this issue:
1)the app is running out of contiguous blocks in the 2G process address space
2)the app is running out of virtual address space period

In either case, having more virtual address space either

· moves the limit that is being hit in 2 above to a higher limit, or

· makes more room for more contiguous blocks and lets the app run longer before running out of contiguous blocks

In either case, when this works as people are reporting it reduces or eliminates the “Out of Memory” issue.

Now a couple extra bits.

Since this is virtual address space we are talking about, yes this can help you regardless of your actual physical RAM.

And this doesn’t conflict with Superfetch on Vista since that is merely the caching system for the virtual memory manager and doesn’t impact the virtual address space available to the app. Superfetch, when it is working properly, does a better job of fulfilling page requests made by the app and means speedier performance of the virtual memory manager.

Comments

• Anonymous
  June 15, 2007
  Thank you Phil for sharing the details of D3DX and its reliance on 31 bit memory addresses. Interesting that this issue hasn't been addressed before. I would've thought that games running under 64-bit Windows would've been bitten by this before? Which reminds me I better go back and see if you've answered any of the "why no 64-bit version of FSX?" questions... Surely this would be the best way to get the party started. Thanks also for sharing the tidbit about PCIExpress memory being mapped into the kernel memory space. I wasn't aware things were different before. But hey... That's why we had the AGP aperture setting in the BIOS setup in the old days, right? -- Rune

• Anonymous
  June 15, 2007
  The comment has been removed

• Anonymous
  June 15, 2007
  Phil, We have a couple users at PMDG I've tried to help who are always seeing these types of errors and I happen to have an identical machine specs wise to one of them and yet I've never been able to reproduce it myself even with the exact same scenario/addons etc. Are you aware of any system-specific issues that can cause the OOM error message?  I'd assumed for quite a while that the message wasn't actually indicative of the system really running out of physical memory since the total usage was usually only slightly above 1GB w/ 2GB physical installed when the error message occurred.  What you're saying here though seems to indicate that the message is only triggered by a real actual OOM situation though right? Ryan

• Anonymous
  June 15, 2007
  Tabs: OOM can be caused by either really being out of virtual address space or fragmentation such that contiguous blocks of size x cannot be allocated, as I mentioned. If an alloc fails, to the app that indicates OOM. From the apps point of view, it is almost not relevant that it is only an alloc of that specific size that fails or really being out of address space - it just knows it couldnt get what it wants. Your case sounds like fragmentation and lack of contiguous memory.

• Anonymous
  June 15, 2007
  "I would've thought that games running under 64-bit Windows would've been bitten by this before?" Games under 64 bit Windows would only run into this issue if they use the IMAGE_FILE_LARGE_ADDRESS_AWARE flag. 32 bit executables with this flag get 4GB address space in Win64, while those without the flag continue to have only 2GB. If the game is 64 bit it would be linked against the 64 bit D3DX which I presume doesn't have this issue. If it's the last issue they're running into (the tagging one) and they do this consistently this would mean they're checking for 0x8000000000000000 as the tag, which is safe since a 64 bit process gets "only" 8TB of address space which is considerably less than that.

• Anonymous
  June 21, 2007
  Although FSX ran fine without the largeadress fix for a while the OOM problems reappeared, why I have no idea.  So went back to this fix since then no more problems and the program runs like a champ at FPS 20 fixed, even in NY city, albeit with no bloom, no shadows, water at 2X low, and road traffic at 12.   All other scenery at the max except autogen one step from the highest.  Hardly any fuzzies.  NVidia card set to max quality on all settings. Interestingly using 2048, or 2560, continued to cause crashes so switched to 3072.  After this no more problems.  Vista reports the memory (with 4 megs of memory installed) as 2816.  After some internet reading on the subject it appears that one should leave some overhead available for the the system.  Presumably entering the memory as 3072 helps in some way.  For someone with 3 megs the number to be entered might be different. Only other issue is that if my daughter plays Sims 2 (with the fix attached) and I play FSX afterwards the program acts strangely.  So a reboot is necessary. The reason for this post is hopefully it will help others. Vista Ultimate 32 bit, 680i NVidia board, 4 megs memory, 6800GTX with latest beta driver.  Overclocked on air to 3600 with clock at 8 and bus at 1800 with memory set at 900.  Very, very stable at these settings but one needs a good power source, 1000w works great. Hope this helps and good luck with DX10 and thank you for the blogs.