Share via

Windows Server: Deduplication Tuning for Deployments with High Saving Rates

  • Article

The default settings of deduplication are normally sufficient for the majority of server deployments.  However, systems with very high deduplication savings rates (for example, over 75%) may benefit from a few manual adjustments to run optimization jobs more efficiently.

First, be sure the November 2014 update rollup for Windows Server 2012 R2 (KB3000850) has been installed on the deduplication server (and on all server nodes in the case of a cluster).  Many critical performance specific fixes (especially for larger scale systems) were included with that update.

Some optimization jobs may be occasionally cancelled early reporting the job "uses too much memory". The following registry key change via PowerShell can be used to increase the memory usage allowed for the optimization job:

Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\ddpsvc\Settings -Name WlmMemoryOverPercentThreshold -Value 1000

In the case of clusters, the following PowerShell command should be run on all nodes:

Set-ItemProperty -Path HKLM:\Cluster\Dedup -Name WlmMemoryOverPercentThreshold -Value 1000

With very high savings rates (>80%), optimization jobs may run slower due to the system needing to allocate more memory to handle compression processing.

The amount of memory assigned to the deduplication job for this purpose can be adjusted to a more appropriate value by using the following PowerShell command.

Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\ddpsvc\Settings -Name EstimatedCompressionRate -Value 5

Again, in the case of clusters the following PowerShell should be run on all nodes:

Set-ItemProperty -Path HKLM:\Cluster\Dedup -Name EstimatedCompressionRate -Value 5

Note for systems with especially high savings rates (>95%), setting EstimatedCompressionRate to 10 would be recommended.

Also, consider turning off the Full Garbage Collection job as detailed in the article: Churn from full garbage collection during deduplication causes performance problems in Windows Server 2012