Eviction Troubleshooting (Windows Server AppFabric Caching)
A Windows Server AppFabric cache cluster uses eviction to control the amount of memory that the Caching Service uses on a cache host. Eviction occurs during two scenarios:
The available physical memory on the server is critically low.
The Caching Service's memory usage exceeds the high watermark for the cache host.
Eviction is not necessarily a problem. If you have an unusually high volume of activity on the cache cluster, eviction can remove the least recently used items and prevent memory problems from causing throttling. For more information about throttling, see Throttling Troubleshooting (Windows Server AppFabric Caching). However, regular eviction runs could indicate a lack of memory or a configuration issue with the cache host or one more caches.
For more information about eviction, see Expiration and Eviction.
Diagnosing Eviction Problems
From an application perspective, eviction causes applications not to find items in the cache that would otherwise be there. This means that the applications must repopulate those items, which could adversely affect application performance.
To assess the frequency of eviction runs, you can use Performance Monitor to log two counters four counters.
| Counter | Description |
|---|---|
AppFabric Caching:Host | Total Eviction Runs |
The total number of eviction runs since the AppFabric Caching Service started. |
AppFabric Caching:Host | Total Data Size Bytes |
The total size of cached data in bytes. |
AppFabric Caching:Host | Total Evicted Objects |
The number of objects evicted since the AppFabric Caching Service started. |
AppFabric Caching:Host | Total Memory Evicted |
The amount of memory in bytes evicted since the AppFabric Caching Service started. |
Because the eviction counters are cumulative since the start of the Caching Service, you must notice changes in this value over time.
There are also two events that relate to eviction runs. These events occur in the Operational log for AppFabric.
| Event ID | Description |
|---|---|
118 |
Service available memory low. |
115 |
Service entered throttled state. |
The 118 event indicates that the Caching Service has reached the high watermark memory level. This triggers an eviction run to attempt to return the memory levels to the low watermark memory level. The 115 event indicates that the server is low on memory and has entered the throttled state. This triggers an eviction run to attempt to resolve the throttled state. For more information about throttling, see Throttling Troubleshooting (Windows Server AppFabric Caching).
Note
For more information about the tools described here, see Health Monitoring Tools (Windows Server AppFabric Caching).
Resolving Eviction Problems
If eviction runs occur because the server is low on memory, then it is likely that the server is also in a throttled state. Fore more information about how to resolve throttling, see Throttling Troubleshooting (Windows Server AppFabric Caching).
If eviction runs occur when the server is not throttled, then review the cache host configuration settings with the Get-CacheHostConfig Windows PowerShell Command. For example, consider the following cache host settings.
Get-CacheHostConfig -HostName CacheServer1 -CachePort 22233
HostName : CacheServer1
ClusterPort : 22234
CachePort : 22233
ArbitrationPort : 22235
ReplicationPort : 22236
Size : 100 MB
ServiceName : AppFabricCachingService
HighWatermark : 90%
LowWatermark : 80%
IsLeadHost : False
In the previous example, the size of memory reserved for the Caching Service on CacheServer1 is only 100 MB. The high watermark level is 90%. This means that when the memory used on this cache host reaches ninety percent of 100 megabytes, the Caching Service will be low on memory and an eviction will be triggered. In this example, the problem is that the memory for the Caching Service was set unusually low. The solution is to increase the amount of caching memory on the CacheServer1 cache host. The following example stops the cache cluster and changes the size to 1000 MB.
Stop-CacheCluster
Set-CacheHostConfig -CacheSize 1000 -HostName CSD1516217 -CachePort 22233
Warning
You should not increase the amount of available memory for the Caching Service unless you also have the available physical memory to support the increase.
Note
The CacheSize combines with the HighWatermark to determine when to trigger eviction runs. It does not guarantee that the cache memory will stay at or below that level. For an example, see the discussion of non-eviction caches in the next section.
Non-Eviction Cache Considerations
By default, new caches enable eviction; however, you can create a cache that does not enable eviction. If eviction is not enabled for a cache, then the eviction runs will not remove any items from that cache. This has several implications:
If the memory pressure is caused by a non-eviction cache, then eviction runs will not resolve the problem. In this case the eviction runs will occur repeatedly.
If the memory pressure is caused by a non-eviction cache, then the eviction runs will remove items from other caches that enable eviction.
To determine which caches enable eviction, you can use the Get-CacheConfig Windows PowerShell command. It is possible to use a simple script to display each cache, its eviction setting, and its size.
$cache = Get-Cache
foreach($cache in $caches) { Write-Host $cache.CacheName (Get-CacheConfig $cache.CacheName).EvictionType (Get-CacheStatistics $cache.CacheName).Size }
The following example shows sample output from this script.
Cache1 None 20481960
Cache2 LRU 20706360
default LRU 4099920
In the previous example output, you can see that Cache1 has disabled eviction (None instead of LRU), and it currently has a size of 20481960 bytes.
See Also
Concepts
Troubleshooting Server Issues (Windows Server AppFabric Caching)