A Primer on Windows' Memory Management

In 1986, Intel released the 386 processor, which offered support for a new instruction set (IA32), that was an extension of the original x86 instruction set. Among the most notable features in IA32 was an increase in the amount of memory the CPU could address, moving from 20bit addressing(1MB) to 32bit addressing(4GB)(ed: this is not including the convoluted mess that was segmented addressing). All x86 CPUs released since then have supported the same instruction set, including the same 4GB limit. Only recently have 64bit x86 CPUs been released, and while they support more than 4GB of memory, still are limited to 4GB when operating in 32bit mode.

As we have mentioned in previous articles, most modern system running in 32bit x86 mode have trouble seeing and using more than roughly 3GB of memory. This is because part of the total 4GB of memory space (not the physical memory) is reserved for various functions, such as computer components transferring data between each other using memory-mapped input-output(MMIO). The textbook example of this is the CPU transferring data to the memory of a video card, where a chunk of the address space equal to the size of the memory of the video card is reserved by the video card, and any data sent to those addresses actually ends up going to the video card. This design has many technical merits, but it makes the consumed memory addresses unavailable for use with physical memory.

Things only get more complex as we start including the operating system (in this case Windows) in to the equation. The above is actually handled by a combination of Windows and the BIOS, meanwhile Windows also needs some address space so that programs can communicate with the Windows kernel, for storing buffers, for storing memory tables, etc; all of which means we have lost even more address space. All of the above besides preventing us from addressing 4GB of physical memory are also the cause of the actual 2GB barrier that is the problem.

Quickly, there is one more pre-requisite piece of information: virtual address space. For a 32bit Windows application(Win32), each application has a full 4GB worth of private addressing space that it can use, which again is 32bits and a result of how a 32bit processor works. How the above exactly works is beyond the scope of this article, but it's sufficient to say that at some point virtual addresses get translated in to other addresses for mapping data between the application and physical memory or the swap file. The important thing to take from this is that each application has its own 4GB virtual address space, regardless of the hardware the computer contains or what applications are running. Now we may begin to understand the 2GB barrier.

Due to design reasons outside the scope of this article, Windows takes a pre-determined portion of each application's virtual address space and reserves it for itself. Windows uses its portion of the virtual address range for all of the address needs listed above. At the end of the day, and what really matters, is that in designing Windows Microsoft opted to split up the virtual address space of an application in half; 2GB goes to Windows (kernel space) and 2GB goes to the application (user space). Under normal circumstances this 2GB of space is all a 32bit application has to work with, this is the 2GB barrier and as we'll see is the cause of the problems with Supreme Commander.

It is also worth noting at this point that virtual address space is not directly correlated to physical memory size. Windows and any applications running under it may use up to all of their allocated virtual address space, with Windows simply swapping data between the hard drive and physical memory if the amount of physical memory needed is in excess of what's available - this is virtual memory, not to be confused with virtual address space. The only meaningful relation between virtual address space and physical memory is that the amount of physical memory an application can use can never be greater than its virtual address space. The user space must take up a larger portion of the virtual address space if an application is to use more than 2GB of physical memory.

Index Removing the 2GB Barrier
Comments Locked

69 Comments

View All Comments

  • johnsonx - Saturday, July 14, 2007 - link

    We are both right. While many 32-bit apps run fine on 64-bit windows, some don't. In particular, any app that employs any sort of driver (I'm talking software drivers, not hardware drivers) must at least have the driver compiled in 64-bit mode. In my case the example is a file synchronizing application that installs a driver to monitor file system changes. There are many apps that wouldn't appear to need a driver, but in fact they do. If you want to see the software drivers on your system, go into Device Manager and select Tools->Show Hidden Devices. Now look in "Non-Plug and Play Drivers". Everything in there is a software driver. If you're running 64-bit Windows, everything in there MUST be 64-bit. Applications can also install drivers at runtime, which also must be 64-bit.

    There are other 32-bit apps that for whatever reason just won't work on 64-bit windows. I believe AutoCAD 2006 and earlier have this problem. Likewise, any apparently 32-bit app that still has some 16-bit legacy code won't run either (for example some rather old but still perfectly useful 32-bit apps use a 16-bit version of InstallShield; the app itself might run, but there's no way to get it installed.).

    Regarding the Retail version, I've never had a retail version of Vista in my hand. I recall Microsoft saying at one point that both versions would be on the same disc and had never heard anything contrary.
  • Christopher1 - Sunday, July 15, 2007 - link

    So this 64-bit software driver problem doesn't apply to things like games, I would assume, considering that they usually do not install software drivers?

    Well then, I think I'm going to contact Toshiba and use their 'Upgrade to x64 version of your operating system' option that recently appeared, if this doesn't effect games.
  • miahallen - Sunday, July 15, 2007 - link

    No, all games should run fine on x64...however, tests have shown that most games run slightly faster on Vista x86.
  • miahallen - Saturday, July 14, 2007 - link

    OK you are right, I stand corrected. I knew about this, just wasn't thinking about it. But from my experiance, those particular apps (that use SW drivers) seem to be the ones getting the quickest attention. I do recall having a couple issues related....a freeware plugin for MCE called mymovies. x64 is not supported at all. Also, Daemon Tools & PowerISO were having issues...but I believe they have both been resolved since then (I went to x86 about four months ago, not much point for me to go back until my next upgrade - 8GB RAM).
  • strikeback03 - Friday, July 13, 2007 - link

    Somehow I'm doubting that the "average computer user" who buys a prebuilt system without knowing or caring about OS design will stand for random peripherals not working due to a 64bit OS. Probably part of why finding 64bit versions of Vista on prebuilt systems is hard. What should happen is Microsoft state right now that the next version of Windows will be 64bit only; then try and educate consumers that some things won't be compatible, and notify companies that they really need to have their 64bit drivers working by then.
  • jay401 - Thursday, July 12, 2007 - link

    No, they should have rendered their units properly (if that's even the real reason... somewhere though they aren't doing something right in their coding, given the way SupCom eats memory).
  • jpeyton - Thursday, July 12, 2007 - link

    On page 5, under Software Test Bed, the RAM should read (4x512MB) instead of (4x512GB).
  • fic2 - Thursday, July 12, 2007 - link

    First sentence on the front page summary:

    "As applications being to approach"

    I assume that "being" is supposed to be begin.

Log in

Don't have an account? Sign up now