Share via

Exchange 2013: Store, FAST, and ESE Cache Demystified…Hopefully!

  • Article

After getting great community response on my Exchange 2013 Mail flow Demystified blog I decided to do a similar one on Store, FAST, and ESE Cache. Much like mail flow a lot has changed with Exchange 2013 in terms of Store. So hopefully I can help clarify exactly what has changed in 2013 with these components. First let’s start with Store since this is the core to your Mailbox Server.

With Exchange 2013 Store.exe got a much needed overhaul (hasn’t changed much since Exchange 4.0) and was rewritten entirely in managed C# code. The  store.exe process is now broken out into 2 processes: Microsoft.Exchange.Store.Service.exe and Microsoft.Exchange.Store.Worker.exe. The Store.Service process is responsible for managing and terminating all of the Store.Worker processes. Each database on the mailbox server will get its own Store.Worker process regardless if it an active or passive database copy. You can see in the image below both processes and how the database WorkerProcessID (PID) correlates to Store.Worker process. By splitting Store into multiple process this minimizes the chance a single database brings down the entire store.exe and crashing\failing over all databases on that server.

Anyone who has worked with Exchange 2010 or older knows if you look at processes, Store.exe took the majority of the server’s memory (75-90%). This by design and is the default behavior. ESE (store.exe) will release memory back to the system if it is needed by another process. You can also set the msExchESEParamCacheSizeMax registry key to specify the max memory ESE can consume on that server. This design worked great for Exchange 2010 and previous versions but due to significant changes in 2013 this process had to get architected.

Due to the fact that Exchange 2013 needs more memory for other processes like Search (FAST) and Transport we had to limit the amount of RAM that ESE used. Now with 2013 ESE (Store) will use 25% of the total RAM in the system. So if your server has 48GB of memory ESE will utilized 12GB of that for cache. Now that 12GB will need to get broken up to support each of the databases hosted on that server. Before I get into the exact formula ESE will use approx. 80% of the 12GB for Active copies and 20% for passive copies. Easy enough right? Not so Fast. There is a problem that can arise if you don’t properly set the MaximumActiveDatabases on each mailbox server (Set-Mailbox –MaximumActiveDatabases). So let’s look at two examples below and you will see exactly what I mean.

Example 1 – Our MBX server has 48GB of memory and hosts 8 databases (4 Active and 4 Passive). In this scenario we didn’t set the MaximumActiveDatabases parameter on the MBX server. You can see that out of 12GB allocated for ESE we are only utilizing 7.2GB in a typical scenario.

Total Server RAM = 48GB
48GB*25%=12GB allocated for ESE cache
There are 4 active copies on the server
There are 4 passive copies on the server (these can potentially become active)
12GB/(4+4) = 1.5GB
Active database cache = 1.5GB
Passive database cache=1.5GB*0.20 = 0.3GB
Max Cache Target for active and passives = 12GB (25% RAM)
Typical (Used) Cache Target = (4*1.5GB + 4* 0.3GB = 7.2GB)

Example 2 – Our MBX server has the same 48GB of memory and still hosts 8 databases (4 Active and 4 Passive). In this scenario we set the MaximumActiveDatabase parameter on the MBX server to 4. Now you can see that in the typical scenario we utilize 12GB (100%) of the allocated ESE Cache.

Total Server RAM = 48GB
MaximumActiveDatabases = 4
48GB*25%=12GB allocated for ESE cache
There are 4 active copies on the server
There are 4 passive copies on the server (these can’t become active because our max is 4)
12GB/(4+4*.20) = 2.5GB
Active database cache = 2.5GB
Passive database cache=2.5GB*0.20 = 0.5GB
Max Cache Target for active and passives = 12GB (25% RAM)
Typical (Used) Cache Target = (4*2.5GB + 4* 0.5GB = 12GB)

You can see how making sure you set the MaximumActiveDatabase in your environment (if required) can make a huge difference. If we are talking about a standard user who utilizes 5MB of ESE Cache this could be the difference of 204 mailboxes more\less per active database. Proper configuration = Optimal performance!!

Lastly I want to talk about the new search platform used by Exchange called FAST. Many will be familiar with the FAST search technology because it is used by SharePoint. I am not going to go into the new Search Infrastructure internals but focus on its processes and memory requirements on the MBX Server. The Microsoft Exchange Search service starts a controller processes named hostcontrollerservice.exe. In turn the Host Controller process starts 4 worker processes named noderunner.exe. You can see both of these processes below. Each of these nodes has a different responsibility in the Search Infrastructure including Content, Query, Index, and Admin. You will notice that these worker process take up a significant amount of memory. As Best Practice plan to allocate approx. 15% of the total system memory to the entire Search Infrastructure\Processes. This means in the examples above we would use 7.2GB for Search.

Comments

  • Anonymous
    January 01, 2003
    @Itworkedinthelab - If you set the max databases to 4 in a 2 node dag with 8 DBs only 4 databases would be mounted on either server at any given time. This would mean if one server failed 4 databases would be offline :)

  • Anonymous
    January 01, 2003
    Although there has been lof of article since the preview on managed store but again refreshing indeed & Thanks!

  • Anonymous
    January 01, 2003
    @itworkedinlab - it could affect performance but 2010 will allocate a specific amount of RAM regardless of databasesusers. More users you add over our recommended will slow derogate the performance.

  • Anonymous
    July 31, 2013
    Hi very nice article just one thing bothers me:) what happens if i set maxdata... to 4 while the server has potentially 8 active databases(lets say its a two node dag) and one node fails and now all 8 are on this one configured with a max of 4 wouldnt that cause a performance degrade instead of performance optimization?

  • Anonymous
    July 31, 2013
    Thanks thats what i thought so performance might be better but u lose availability usuarly in 2010 i would set it when sizing was a factor(for example server hardware couldnt handle more than 4000 users=16 database,just an example) this doesnt affect exchange 2010 ese cache right?

  • Anonymous
    July 03, 2014
    Great article! "Set-MailboxServer –MaximumActiveDatabases" I suppose.:)

  • Anonymous
    November 26, 2014
    Great read, well written.

  • Anonymous
    June 11, 2015
    Confused on the calculations you are using. You didn't describe a passive database cache utilizing 20% of the active database cache size, but that is what you use in your second scenario calc. The description says 25% of total system RAM partitioned as 80% for active and 20% for passive. Why does setting the MaxActiveDB to a lower number of active DBs allowed increase the utilization of RAM.

  • Anonymous
    June 11, 2015
    Saw this post which helps. http://blogs.technet.com/b/samdrey/archive/2014/02/03/exchange-2013-memory-ram-allocation-by-the-store-with-an-example.aspx