Adjusting Buffer Settings for Event Tracing for Windows (ETW)

We instrumented The Concurrency Visualizer within Visual Studio 2010’s profiler via Event Tracing for Windows (ETW), which depends on a number of buffers to cache data before writing it to disk. If The Concurrency Visualizer complains of lost kernel and / or user mode events during creation of a profile report, default settings for these ETW buffers may be too low for your system or application.

Symptoms:

• When kernel mode events are lost, we report a critical failure and then stop. You will not get a report, as we are unable to render one when these events are missing.

 

• With lost user mode events, we also show an error. However, we still produce a report because the problem is less severe. As is noted in the screen shot below, within your report visible symptoms of this problem may include loss of one or more of your custom scenario markers, or of our own markers that identify Parallel Extensions constructs within your code such as Parallel.For or parallel LINQ.

Prevention:

It is usually possible, via upward adjustment of a few key ETW buffer settings, to reduce the risk of losing events and ensure a complete report. However, as buffer settings are increased, memory consumption is also increased so adjust these values carefully to ensure optimal performance.

For the shipping version of VS2010, we expect to increase our default ETW buffer values relative to Beta 2. We very rarely see dropped events with these increased settings, and performance continues to be good. So, we’re sharing this information now to help you optimize settings for Beta 2 and ensure that you enjoy consistent success in profiling your applications with our tool.

Please note that if event loss is reported during report analysis and creation, those events were actually lost during profiling. Thus, adjusting ETW settings and then re-opening an existing profile report will return the same errors. To eliminate these errors, you must re-profile your application after adjusting your ETW buffer settings.

The ETW buffer values that may be modified are shown in the table below. That table shows default values used in Beta2 and the increased default values that we chose, based on customer feedback, for the released product. If you are running Beta2, we strongly recommend creating registry settings that match the planned default values for the release version. For those of you who want to know exactly what you are adjusting, definitions for each of the values below are available in the following MSDN article: https://msdn.microsoft.com/en-us/library/aa363784(VS.85).aspx.

To change the default settings noted below, create a registry key at HKCU\SOFTWARE\Microsoft\VisualStudio\10.0\VSPerf\Monitor\EtwConfig, with the following DWORD entries:

Value	Type	Default Settings for Beta2	Default Settings planned for Release
FlushTimer	DWORD	1	0
BufferSize	DWORD	128	256
MinBuffers	DWORD	256	512
MaxBuffers	DWORD	1024	1024

We have seen one case where a customer observed lost events despite the updated settings above. That customer was able to eliminate loss of events by increasing MaxBuffers to 2048. If lost event problems persist after verifying the settings above, it may be useful to increase the MaxBuffers setting as noted.

Bill Colburn - Parallel Computing Platform

Comments

• Anonymous
  January 12, 2011
  I generated a system health report with advanced systems tools this morning and received an error warning: Symptom: Missing Events in Event Log Details: Investigate why 13% (2,607) events were lost during data collection. The settings for even tracing for windows (ETW) maximum buffers and buffer size may not be optimal depending on which data sets are being              collected. I've never received this before. OS is Vista Home Prem 64 bit How do I correct this?

• Anonymous
  January 12, 2011
  I generated a system health report with advanced systems tools this morning and received an error warning: Symptom: Missing Events in Event Log Details: Investigate why 13% (2,607) events were lost during data collection. The settings for even tracing for windows (ETW) maximum buffers and buffer size may not be optimal depending on which data sets are being              collected. I've never received this before. OS is Vista Home Prem 64 bit How do I correct this? metrotek2010@live.com

• Anonymous
  January 07, 2012
  I have the same error report i use windows7 64 bit . I can not find the buffer to adjust the level

• Anonymous
  January 09, 2012
  Is this a problem you are seeing in the Concurrency Visualizer?  If so, the entry above describes the process of increasing the ETW buffer sizes by modifying the registry.  Try adjusting the buffer sizes to the maximum sizes described in this article: msdn.microsoft.com/.../aa363784(v=vs.85).aspx.   Based on your comment, it sounds like this might be an issue you are seeing with HP's system health check suite of tools.  If this is the case, I am not in a position to provide help with that product.

• Anonymous
  September 16, 2012
  Kind of scary how often I am seeing the System Diagnostics in Windows being brushed off as third-party health check/other health check suites.  Do people just not know these tools exist?

• Anonymous
  June 02, 2014
  The comment has been removed

• Anonymous
  May 10, 2015
  Nobody can help, I have been periodically searching for a solution for over two years!

• Anonymous
  February 20, 2016
  Investigate why 53% (21,213) events were lost during data collection. The settings for Event Tracing for Windows (ETW) maximum buffers and buffer size may not be optimal depending on which data sets are being collected.