How to stress and measure the Garbage Collector
Many years ago, I was writing real time applications for monitoring sound signals bouncing off submarines. That code (written in Fortran and PDP-11 assembly code) had to process the incoming sound at the rate of thousands of data points per second, doing real-time Fast Fourier Transforms and analyzing the results, printing graphs. Using a managed language, with unpredictable garbage collection can make this much more problematic. The CLR needs to suspend managed threads for garbage collection, which can cause slowdowns in user code, causing interruptions in the timing, perhaps causing missed data points.
How can one write very little simple code with a huge impact on the .Net garbage collector?
You’d think one way to do this is to consume large amounts of memory. That’s one way, but here’s another:
Thinking about GC, here’s a simple model of how it could work. Imagine every object is a person in a room:
1. Suspend all threads, so that object references are constant
2. Tell every person to raise a hand
3. Define a “GC Root” person: one who cannot be freed. Examples are
a. static vars, globals
b. Variables on a stack
c. Interop wrappers
4. For each “GC Root”:
a. Recursive step:
i. Put your hand down (I still need you as non-garbage)
ii. Tell each object that you reference (point to) to
1. Do the recursive step
5. Those with their hands still up are “garbage” and can be collected
6. Optionally compress/consolidate free space in the room by moving objects
a. Moving an object changes its objectId, so all references need to be patched too.
So creating an example that takes a while might be to create a long chain of references: A=>B=>C=>D
Here’s some code to generate a long object reference chain.
class MyLinklist
{
public MyLinklist _next;
}
MyLinklist _root = null; // static root
for (int i = 0; i < nIter; i++)
{
var x = new MyLinklist
{
_next = _root
};
_root = x;
}
_root = null;
How do I measure the effectiveness of this code on GC? I need some way of determining the performance impact of the code.
I wrote a post https://blogs.msdn.microsoft.com/calvin_hsia/2014/05/29/how-to-monitor-and-respond-to-memory-use/ a few years ago, which showed how to get performance counter numbers.
I combined that with the graphing code in https://blogs.msdn.microsoft.com/calvin_hsia/2017/08/31/graph-poker-hand-distributions/, resulting in the sample code below.
File->new->Project->C#->WPF App "GCStress"
Project->Add Reference to System.Drawing, WindowsFormsIntegration, System.Windows.Forms, System.Windows.Forms.DataVisualizetion
Paste in the code below to replace the code in MainWindow.xaml.cs
The perf counter monitoring code runs and samples the perf counters periodically and updates a moving graph. The GC Stress code can be turned on/off with the button.
When you run the code, it starts monitoring the selected performance counters. Of course, you can use the PerfMon application built into Windows already https://msdn.microsoft.com/en-us/library/aa266946%28v=vs.60%29.aspx?f=255&MSPPError=-2147217396 , but that is quite general purpose and not easily customizable.
Click the Go button to start stressing the GC. From the graphs, the % of time spent in GC is > 60%
You can choose various performance counters to monitor, and you can specify various parameters for the performance graph
Try running the code, and just move the window around, or resize, monitoring how ProcessorPctTime changes.
Try increasing the NumIterations, watching the ProcessorVirtualBytes. Eventually the program will slow down and crash Out of Memory.
See also
https://blogs.msdn.microsoft.com/calvin_hsia/2005/09/16/the-fast-fourier-transform/
https://blogs.msdn.microsoft.com/calvin_hsia/2011/10/31/more-fun-with-the-fast-fourier-transform/
<Code>
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Forms.DataVisualization.Charting;
using System.Windows.Forms.Integration;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.Windows.Threading;
/*
File->new->Project->C#->WPF App "GCStress"
Project->Add Reference to System.Drawing, WindowsFormsIntegration, System.Windows.Forms, System.Windows.Forms.DataVisualizetion
*/
namespace GCStress
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
TextBox _txtStatus;
Chart _chart;
/// <summary>
/// PerfCounters updated periodically. Safe to change without stopping the monitoring
/// </summary>
public int UpdateInterval { get; set; } = 200;
public int NumIterations { get; set; } = 3000;
public int NumDataPoints { get; set; } = 100;
public bool ScaleByteCounters { get; set; } = true;
public bool SetMaxGraphTo100 { get; set; } = true;
public PerfCounterType PerfCounterTypesToShow { get; set; } = PerfCounterType.ProcessorPctTime;
[Flags] // user can select multiple items. (beware scaling: pct => 0-100, Bytes => 0-4G)
public enum PerfCounterType
{
None,
ProcessorPctTime = 0x1,
ProcessorPrivateBytes = 0x2,
ProcessorVirtualBytes = 0x4,
GCPctTime = 0x8,
GCBytesInAllHeaps = 0x10,
GCAllocatedBytesPerSec = 0x20,
}
Dictionary<PerfCounterType, string> _dictPerfCounterDefinitions = new Dictionary<PerfCounterType, string>
{
{PerfCounterType.ProcessorPctTime,"Process|% Processor Time|ID Process" },
{PerfCounterType.ProcessorPrivateBytes,"Process|Private Bytes|ID Process" },
{PerfCounterType.ProcessorVirtualBytes,"Process|Virtual Bytes|ID Process" },
{PerfCounterType.GCPctTime,".NET CLR Memory|% Time in GC|Process ID" },
{PerfCounterType.GCBytesInAllHeaps,".NET CLR Memory|# Bytes in all Heaps|Process ID" },
{PerfCounterType.GCAllocatedBytesPerSec,".NET CLR Memory|Allocated Bytes/sec|Process ID" },
};
CancellationTokenSource _ctsGCStress;
CancellationTokenSource _ctsPcounter;
Task _tskDoPerfMonitoring;
public MainWindow()
{
InitializeComponent();
this.Loaded += MainWindow_Loaded;
}
private async void MainWindow_Loaded(object sender, RoutedEventArgs e)
{
try
{
this.DataContext = this;
this.Title = "GCStress";
this.Height = 600;
this.Width = 1000;
var sp = new StackPanel() { Orientation = Orientation.Vertical };
var spControls = new StackPanel() { Orientation = Orientation.Horizontal };
sp.Children.Add(spControls);
spControls.Children.Add(new Label() { Content = "Update Interval", ToolTip = "MilliSeconds" });
var txtUpdateInterval = new TextBox() { Width = 50, Height = 20, VerticalAlignment = VerticalAlignment.Top };
txtUpdateInterval.SetBinding(TextBox.TextProperty, nameof(UpdateInterval));
spControls.Children.Add(txtUpdateInterval);
var lbPCounters = new ListBox()
{
Width = 150,
SelectionMode = SelectionMode.Multiple,
Margin = new Thickness(10, 0, 0, 0)
};
lbPCounters.ItemsSource = ((PerfCounterType[])Enum.GetValues(typeof(PerfCounterType))).Skip(1); // skip the "None"
lbPCounters.SelectedIndex = 0;
lbPCounters.SelectionChanged += async (ol, el) =>
{
lbPCounters.IsEnabled = false;
AddStatusMsg($"Setting counters to {this.PerfCounterTypesToShow.ToString()}");
// cancel the perf monitoring
_ctsPcounter?.Cancel();
// before we wait for cancel to finish we can do some work
PerfCounterType pcEnum = PerfCounterType.None;
foreach (var itm in lbPCounters.SelectedItems)
{
pcEnum |= (PerfCounterType)Enum.Parse(typeof(PerfCounterType), itm.ToString());
}
// wait for it to be done cancelling
if (_tskDoPerfMonitoring != null)
{
await _tskDoPerfMonitoring;
}
await Task.Run(() =>
{
// run on threadpool thread
lock (_lstPerfCounterData)
{
this.PerfCounterTypesToShow = pcEnum;
ResetPerfCounterMonitor();
}
});
AddStatusMsg($"SelectionChanged done");
lbPCounters.IsEnabled = true;
};
spControls.Children.Add(lbPCounters);
var chkSetMaxGraphTo100 = new CheckBox()
{
Content = "SetMaxGraphTo100",
ToolTip = "Set Max Y axis to 100. Else will dynamically rescale Y axis",
Height = 20,
VerticalAlignment = VerticalAlignment.Top,
Margin = new Thickness(10, 0, 10, 0)
};
chkSetMaxGraphTo100.SetBinding(CheckBox.IsCheckedProperty, nameof(SetMaxGraphTo100));
spControls.Children.Add(chkSetMaxGraphTo100);
var chkScaleByteCtrs = new CheckBox()
{
Content = "Scale 'Byte' counters",
ToolTip = "for Byte counters, scale to be a percent of 4Gigs",
Height = 20,
VerticalAlignment = VerticalAlignment.Top
};
chkScaleByteCtrs.SetBinding(CheckBox.IsCheckedProperty, nameof(ScaleByteCounters));
spControls.Children.Add(chkScaleByteCtrs);
spControls.Children.Add(new Label()
{
Content = "NumDataPoints",
ToolTip = "Number of Data points (x axis). Will change on next Reset"
});
var txtNumDataPoints = new TextBox() { Width = 50, Height = 20, VerticalAlignment = VerticalAlignment.Top };
txtNumDataPoints.SetBinding(TextBox.TextProperty, nameof(NumDataPoints));
spControls.Children.Add(txtNumDataPoints);
spControls.Children.Add(new Label()
{
Content = "NumIterations",
ToolTip = "Number of linklist items (will be multiplied by 1000)"
});
var txtNumIter = new TextBox() { Width = 50, Height = 20, VerticalAlignment = VerticalAlignment.Top };
txtNumIter.SetBinding(TextBox.TextProperty, nameof(NumIterations));
spControls.Children.Add(txtNumIter);
var btnGo = new Button()
{
Content = "_Go",
ToolTip = "Stress GC",
Width = 30,
Height = 20,
VerticalAlignment = VerticalAlignment.Top
};
spControls.Children.Add(btnGo);
_chart = new Chart()
{
Width = 200,
Height = 100,
Dock = System.Windows.Forms.DockStyle.Fill
};
var wfh = new WindowsFormsHost()
{
Child = _chart
};
sp.Children.Add(wfh);
_txtStatus = new TextBox()
{
IsReadOnly = true,
VerticalScrollBarVisibility = ScrollBarVisibility.Auto,
HorizontalScrollBarVisibility = ScrollBarVisibility.Auto,
IsUndoEnabled = false,
FontFamily = new FontFamily("Courier New"),
FontSize = 10,
Height = 200,
MaxHeight = 200,
HorizontalContentAlignment = HorizontalAlignment.Left
};
sp.Children.Add(_txtStatus);
this.Content = sp;
var isGoing = false;
btnGo.Click += async (ob, eb) =>
{
isGoing = !isGoing;
if (isGoing)
{
btnGo.Content = "_Stop";
AddStatusMsg("Starting");
_ctsGCStress = new CancellationTokenSource();
await DoGCStressAsync();
// when the task finishes, then we're stopping
isGoing = false;
}
AddStatusMsg("Stopping");
_ctsGCStress.Cancel();
btnGo.Content = "_Go";
};
await Task.Run(() =>
{
ResetPerfCounterMonitor();
});
// btnGo.RaiseEvent(new RoutedEventArgs(Button.ClickEvent, this));
}
catch (Exception ex)
{
this.Content = ex.ToString();
}
}
class MyLinklist
{
public MyLinklist _next;
//public int[] _array;
//public MyLinklist()
//{
// _array = new int[100];
//}
}
MyLinklist _root = null; // static root
private async Task DoGCStressAsync()
{
await Task.Run(() =>
{
AddStatusMsg($"starting {nameof(DoGCStressAsync)}");
try
{
while (!_ctsGCStress.Token.IsCancellationRequested)
{
int nIter = 1000 * NumIterations;
for (int i = 0; i < nIter; i++)
{
var x = new MyLinklist
{
_next = _root
};
_root = x;
}
_root = null;
}
}
catch (OutOfMemoryException ex)
{
_root = null;
AddStatusMsg($"{nameof(DoGCStressAsync)} {ex.ToString()}");
}
AddStatusMsg($"cancelling {nameof(DoGCStressAsync)}");
});
}
// list of current set of perfcounters we're monitoring
List<PerfCounterData> _lstPerfCounterData = new List<PerfCounterData>();
// use a circular buffer to store samples.
// dictionary of sample # (int from 0 to NumDataPoints) =>( List (PerfCtrValues in order)
public Dictionary<int, List<uint>> _dataPoints = new Dictionary<int, List<uint>>();
int _bufferIndex = 0;
public class PerfCounterData
{
public PerfCounterType pcntrType;
public PerformanceCounter performanceCounter;
public PerfCounterData(PerfCounterType pcntrType, PerformanceCounter performanceCounter)
{
this.pcntrType = pcntrType;
this.performanceCounter = performanceCounter;
}
}
void ResetPerfCounterMonitor()
{
var pid = Process.GetCurrentProcess().Id;
lock (_lstPerfCounterData)
{
AddStatusMsg($"{nameof(ResetPerfCounterMonitor)} {PerfCounterTypesToShow.ToString()} ");
_lstPerfCounterData.Clear();
_dataPoints.Clear();
_bufferIndex = 0;
foreach (PerfCounterType pcntrType in Enum.GetValues(typeof(PerfCounterType)))
{
if (pcntrType != PerfCounterType.None)
{
if (PerfCounterTypesToShow.HasFlag(pcntrType))
{
var pcounter = GetPerfCounter(pcntrType, pid);
var pcntrData = new PerfCounterData(pcntrType, pcounter);
_lstPerfCounterData.Add(pcntrData);
}
}
}
for (int i = 0; i < NumDataPoints; i++)
{
var emptyList = new List<uint>();
_lstPerfCounterData.ForEach((s) => emptyList.Add(0));
_dataPoints[i] = emptyList;
}
}
DoPerfCounterMonitoring();
}
void AddDataPoints(List<uint> lstNewestSample)
{
_dataPoints[_bufferIndex++] = lstNewestSample;
if (_bufferIndex == _dataPoints.Count)
{
_bufferIndex = 0;
}
var task = this.Dispatcher.BeginInvoke(
new Action(() =>
{
// this needs to be done on UI thread
_chart.Series.Clear();
_chart.ChartAreas.Clear();
var chartArea = new ChartArea("ChartArea");
_chart.ChartAreas.Add(chartArea);
int ndxSeries = 0;
foreach (var entry in lstNewestSample)
{
var series = new Series
{
ChartType = SeriesChartType.Line
};
_chart.Series.Add(series);
for (int i = 0; i < _dataPoints.Count; i++)
{
var ndx = _bufferIndex + i;
if (ndx >= _dataPoints.Count)
{
ndx -= _dataPoints.Count;
}
var dp = new DataPoint(i + 1, _dataPoints[ndx][ndxSeries]);
series.Points.Add(dp);
}
ndxSeries++;
if (SetMaxGraphTo100)
{
_chart.ChartAreas[0].AxisY.Maximum = 100;
}
}
_chart.DataBind();
}));
}
void DoPerfCounterMonitoring()
{
_ctsPcounter = new CancellationTokenSource();
_tskDoPerfMonitoring = Task.Run(async () =>
{
try
{
while (!_ctsPcounter.Token.IsCancellationRequested)
{
var sBuilder = new StringBuilder();
var lstPCData = new List<uint>();
lock (_lstPerfCounterData)
{
foreach (var ctr in _lstPerfCounterData)
{
var pcntr = ctr.performanceCounter;
var pcValueAsFloat = pcntr.NextValue();
uint pcValue = 0;
if (ctr.pcntrType.ToString().Contains("Bytes") && !ctr.pcntrType.ToString().Contains("PerSec") && this.ScaleByteCounters)
{
pcValue = (uint)(pcValueAsFloat * 100 / uint.MaxValue); // '% of 4G
}
else
{
pcValue = (uint)pcValueAsFloat;
}
sBuilder.Append($"{pcntr.CounterName}={pcValue:n0} ");
lstPCData.Add(pcValue);
}
AddDataPoints(lstPCData);
}
AddStatusMsg($"Sampling {sBuilder.ToString()}");
await Task.Delay(UpdateInterval, _ctsPcounter.Token);
}
}
catch (TaskCanceledException)
{
}
AddStatusMsg($"cancelling {nameof(DoPerfCounterMonitoring)}");
});
}
public PerformanceCounter GetPerfCounter(PerfCounterType pcntr, int vspid)
{
var strvalues = _dictPerfCounterDefinitions[pcntr].Split("|".ToCharArray());
var perfcountCat = strvalues[0];
var perfcountName = strvalues[1];
var pidstr = strvalues[2];
PerformanceCounter pc = null;
var category = new PerformanceCounterCategory(perfcountCat);
foreach (var instanceName in category.GetInstanceNames()) // exception if you're not admin or "Performance Monitor Users" group (must re-login)
{
using (var cntr = new PerformanceCounter(category.CategoryName, pidstr, instanceName, readOnly: true))
{
try
{
var val = (int)cntr.NextValue();
if (val == vspid)
{
pc = new PerformanceCounter(perfcountCat, perfcountName, instanceName);
break;
}
}
catch (Exception)
{
// System.InvalidOperationException: Instance 'IntelliTrace' does not exist in the specified Category.
}
}
}
return pc;
}
void AddStatusMsg(string msg, params object[] args)
{
if (_txtStatus != null)
{
// we want to read the threadid
//and time immediately on current thread
var dt = string.Format("[{0}],{1,2},",
DateTime.Now.ToString("hh:mm:ss:fff"),
Thread.CurrentThread.ManagedThreadId);
_txtStatus.Dispatcher.BeginInvoke(
new Action(() =>
{
// this action executes on main thread
var str = string.Format(dt + msg + "\r\n", args);
// note: this is a memory leak: can clear periodically
_txtStatus.AppendText(str);
_txtStatus.ScrollToEnd();
}));
}
}
}
}
</Code>