Not all 106 are created equal

Of course that's not accurate. When MSExchange Common source writes an event 106 about performance counters, the format of each event is exactly the same and reads:

 ID:       106
Level:    Error
Source:   MSExchange Common
Machine:  server1.contoso.com
Message:  Performance counter updating error. Counter name is <counter name here>, category name is <category name here>. Optional code: <code here>. Exception: <actual exception here>...<call stack and list of PID - process>

What I mean by the title, is that depending on the reported Exception message, the actual issue and its consequences could be profoundly different. Understanding some of those differences helps on assessing the issue at hand.

 

There are at least 3 common exception messages, these are examples

(Counter, Category, and Exception message vary):

 Performance counter updating error. Counter name is Percentage of MSAUserNetID Cache hits for last minute, category name is MSExchange Global Locator Processes. Optional code: 3. Exception: The exception thrown is : System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.

This is the most common exception and normally not affecting more than the counter itself. Depending on the counter being reported the root cause can be code trying to access counters on an instances that is no longer running or disabled. The call stack could give us more insight on the specific situation.

 

 Performance counter updating error. Counter name is Time in Resource per second, category name is MSExchange Activity Context Resources. Optional code: 2. Exception: The exception thrown is : System.InvalidOperationException: Instance 'ad-umworkerprocess-umworkerprocess.exe' already exists with a lifetime of Process.  It cannot be recreated or reused until it has been removed or until the process using it has exited.

This is normally thrown by MSExchange Activity Context Resources and related to processes running multiple instances with the same name on the server (i.e. microsoft.exchange.store.worker.exe , umworkerprocess.exe, etc.). In this case, a conflict with the name of the counter instance causes only the first counter to be created and the subsequent counter instances to fail. This issue has been internally reported and likely to be fixed in upcoming  CUs.

 

 Performance counter updating error. Counter name is Cache misses in the DistributionListMembership cache/sec, category name is MSExchangeIS Store. Optional code: 2. Exception: The exception thrown is : System.InvalidOperationException: Custom counters file view is out of memory.

This exception message is particularly important because it seems to be the only one with real impact beyond the counter itself. The issue comes when the category MSExchangeIS Store is affected. If Exchange is running in a DAG environment, failing to update any database instance of that counter category, can cause problems activating databases with an error similar to:

 Log Name:      Microsoft-Exchange-HighAvailability/Operational
Source:        Microsoft-Exchange-HighAvailability
Event ID:      306
Task Category: Database Action
Level:         Information
Keywords:      
User:          SYSTEM
Computer:      server2.contoso.com
Description:
Initiating move for database 'DB1' (FromServer=server2.contoso.comm, ToServer=, MoveComment=UnexpectedDismount)

This happens when Active Manager is unable to read the ProcessID counter from MSExchagneIS Store on the pasive database instance. If you're wondering whether you're running into this problem, check on the MSExchangeIS Store counters in perfmon and verify if they have values other than zero for all the database instances.

If the exception message for MSExchangeIS Store is "System.InvalidOperationException: Custom counters file view is out of memory" it is possible that the custom value FileMappingSize is missing from the service's registry key.
This value is created by Exchange Setup when the counters are registered, and it is probably gone if the counters were recreated on your server without specifying this FileMappingSize value (i.e. if a script has been used to reload all Exchange counters in bulk).

The value for Exchange 2013 and 2016 (CU13 and CU2 respectively at the time of this writing) is:

 FileMappingSize : 5242880 (DWord)
PSPath          : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\services\MSExchangeIS Store\Performance

It this value is missing, it can be manually created in the registry . Restart the server to make sure the value is read when each store worker instance starts.

Comments

• Anonymous
  July 09, 2016
  Thanks