Domino memory limitations on Windows

Thursday 21st April, 2005
My RAMdrive topic has prompted a few discussions around the memory limitation of Domino on Win32.

As I understand it, a 32-bit application is unabled to address any memory above 2GB although Windows can. If written to utilise it, an application can utilise this higher memory area in a similar way that DOS apps used HIMEM.

So, as I understand it, Domino is only a 32-bit application and can only utilise 2GB of RAM at OS level. Whilst it consists of many different .exe file tasks, each task can only utilise that first 2GB of available RAM and so they can't use 2GB per task.

Can anybody shed any light on the accuracy of this? I know in practice it certainly doesn't use more thn 2GB, but why doesn't it and why can't it?

  1. 1) Richard Schwartz Said: (21/04/2005 14:28:40 GMT) Gravatar Image
    Domino memory limitations on Windows

    This is a fairly complex question. By that I mean that the answer is not far removed from voodoo. But here's what I believe is true, based on reading lots of reference pages on the MS web site, based on a lot of discussions, based on some quick tests that I ran using the visual studio debugger to explore the memory map of a typical Notes API-based program, and based on some better experiments run by Daniel Nashed:

    There are several limits to consider. One, of course, is physical RAM. The next is the total available swap space. The next is the virtual memory space available to each process. The last is the limit on how much of a process' virtual address space can be allocated as shared memory for use by multiple processes.

    The physical RAM limit is (mostly) irrelevant to this discussion. Domino can use more memory than the server has installed.

    The swap space limit provides an upper limit on the total of all private memory allocated by all processes, plus all the shared memory allocated by all processes. Domino can't use more memory than the size of the swap file allows. Note that this excludes the memory allocated for executable instructions, as these are mapped from their respective EXE and DLL files, not from swap.

    The per-process virtual address space limit is 4 GB. I.e., a 32 bit pointer can identify up to 4 GB worth of virtual addresses. Each process has its own unique address space, but ony a portion of that actually refers to private memory. 2 GB of that address space is pre-allocated to the OS itself. (I'm ignoring boot and/or compile-time switches that can reduce this to 1 GB.) That leaves 2 GB of address space for the application running in the process. That 2 GB must hold all of the application's executable instructions as well as all of its private and all of its shared memory. Notes and Domino are big applications. You can be quite sure that a substantial part of that 2 GB, therefore, is devoted to instructions, but I don't know how much that is going to be. Whatever is left of the 2 GB virtual address space for a process is available for private stack or heap allocation, or for mapping shared memory segments. Domino uses a substantial amount of shared memory, by the way, so each process that uses the Notes API is going to be sharing the same NSF_Buffer_Pool and various other pieces of shared memory. Whatever is left will be available for private stack or heap. In tests run by Daniel Nashed -- in which he loaded a specially designed server task to just grab as much private memory as possible -- the limit on private stack and heap appeared to be about 1 GB.

    The shared memory limit is, AFAIK 2 GB across all processes. The previously mentioned test run by Daniel confirmed that.

    So, here's the thing: the 2 GB limit is fiction, but for all practical purposes it seems to turn out that Domino isn't going to exceed that. If you load enough server tasks, including some custom tasks that are very hungry for private memory, you can exceed 2 GB of memory allocated for a Domino server. But in an ordinary Domino configuruation, this is quite unlikely. About 1.5 GB seems to be the practical limit most people have observed for normal Domino servers on Windows.

    One reason for this is, I believe, is that Domino's default allocation size for some of the things it puts in shared memory (e.g., NSF_Buffer_Pool) are based on the amount of physical memory in the machine! Domino simply does not want to use so much memory that swapping will start, and this is how it tries to guard against it. So, even on systems with 2 GB or 3 GB of physical RAM, Domino is not, by default, going to take full advantage of the available 2 GB of shared memory space across all processes. And since none of the standard server processes use huge amounts of private stack and heap memory, the total across all processes in one server tends to stay well below 2 GB.

    -rich

  2. 2) Bruce Currier Said: (21/04/2005 15:43:37 GMT) Gravatar Image
    Domino memory limitations on Windows

    What if you have a partitioned Domino Se4rver? Would this 2GB apply to each partition or all of them together?

  3. 3) Richard Schwartz Said: (21/04/2005 16:13:34 GMT) Gravatar Image
    Domino memory limitations on Windows

    Also a complex question, but the short answer is that multiple partitions don't share memory with each other, so together mulitple partitions can indeed use more memory than one partition. But there's still the 2 GB per system limit on total shared memory, so at some point you're going to hit the wall. OTOH, none of what I've said necessarily applies to the "Data Center" editions of Windows. Memory management and limits are likely different there, but I don't have details.

    -rich

  4. 4) Richard Schwartz Said: (25/04/2005 13:15:49 GMT) Gravatar Image
    Domino memory limitations on Windows

    I need to correct what I wrote above. I now believe that shared memory comes from within the 2 GB virtual address space allocated to the OS, not from within the uers-mode space, so each task should have access to 2 GB for executables plus stack and heap, and whatever shared memory it allocates.

    -rich

Add Comment
 
Subject:
   
Name:
E-mail:
Web Site:
 
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?