Optimize for the Hot path

The Pareto principle (also known as the 80-20 rule, the law of the vital few, and the principle of factor sparsity) states that, for many events, roughly 80% of the effects come from 20% of the causes. This applies to programming as well: 80% of the people use 20% of the features, 80% of the execution paths go through 20% of the code etc.

Inside a program, the code paths that are executed most often belong to the hot path (with the remaining code paths being on the cold path). One can look at this at multiple levels: instruction, language statements, features. Over the years, multiple studies have shown how to optimize software for the hot path execution.

Here, I want to focus on the developer that’s writing the code. From the start, the developer can easily identify some cold code (for example, the exception handling code). Keeping in mind the usage pattern, the developer can improve the program so it runs faster most of the times.

Here is an example: let’s say that your application (called CalculatorService) keeps track of the number of Add or Subtract operations by incrementing performance counters. It expects that a performance counter category called CalculatorService already exists (for example, the installer of the service creates it). This is a requirement because it runs as Network Service, while creating the category takes Administrator privileges. Inside CalculatorService category we have 2 counters, AddCounter and SubtractCounter. If the category is not installed, CalculatorService must not throw – it just won’t save the counters.

You decide to create a class PerfCounterPublisher that deal with the performance counters. The service class keeps an instance of PerfCounterPublisher and calls methods on it when Add/Subtract operations are executed.

 class CalculatorService
{
    PerfCounterPublisher publisher;

    public CalculatorService()
    {
        this.publisher = new PerfCounterPublisher("CalculatorService");
    }

    public int Add(int a, int b)
    {
        this.publisher.IncrementCounter("AddCounter");
        return a + b;
    }

    public int Subtract(int a, int b)
    {
        this.publisher.IncrementCounter("SubtractCounter");
        return a - b;
    }
}

A simple implementation of the PerfCounterPublisher can look like this:

 class PerfCounterPublisher
{
    string categoryName;

    public PerfCounterPublisher(string categoryName)
    {
        this.categoryName = categoryName;
    }

    public bool IncrementCounter(string counterName)
    {
        if (PerformanceCounterCategory.Exists(this.categoryName))
        {
            PerformanceCounter counter = new PerformanceCounter(
                this.categoryName,
                counterName,
                false);
            counter.Increment();
            return true;
        }

        return false;
    }
}

So, if the category is installed, we increment the counter; otherwise, we just return false, letting the user know that the counter wasn’t incremented.

This code can be improved. PerformanceCounterCategory.Exists is a rather expensive operation. If we look at usage pattern, most of the time the counters are installed. So we penalize most of the scenarios by adding the check.

How can we optimize the code for the hot path?

Since we expect the category to exist, we will try to create the counter without the check; in the rare cases where the category doesn’t exist, creating the counter will throw InvalidOperationException – in this case, we catch the exception and return false (counters not incremented). IncrementCounter looks like this:

 public bool IncrementCounter(string counterName)
{
    try
    {
        PerformanceCounter counter = new PerformanceCounter(
            this.categoryName,
            counterName,
            false);
        counter.Increment();
        return true;
    }
    catch (InvalidOperationException)
    {
        return false;
    }
}

Depending on the application scenario, we can make further improvements. For example, if we don’t expect the category is installed after the service has been started, we can check if the category exists in the constructor and set a flag to disabled if it doesn’t. Or we can postpone the check to the first time a call to IncrementCounter is done and set the flag there. If the flag has been set, future IncrementCounter become NoOp.

This is a very simple example, but I hope it expresses what I tried to convey. When writing code, make sure you understand how it’s going to be used, identify the how path based on this and always optimize for the hot path.