Learning to Profile

I went to a meeting with Rico the other day and he showed us a few approaches he uses when solving performance issues. He is a performance engineer with many years of experience so it really was a case of watch and learn. This got me thinking about how people can best learn to use performance tools.

One starting point in this process is to consider my own experience learning a more mature dynamic code analysis tool - the debugger. Think back to the first time you ever started up a program running under a debugger. What was the first thing you did? My first debugging experience went something like this:

• Set a breakpoint at the beginning of main() - this was C/C++ afterall.
• Run the code in the debugger. Hey, it stopped. cool.
• Step through a few lines of code and inspect the values of some local variables.

Sit back and think that's pretty cool - maybe I'll have to use a few less printfs to work out what's going on with my program. That's pretty much it. Gradually I learnt more and more about things like:

• The difference between Step In, Step Over, Step Out, Run to Cursor
• The value of different types of breakpoints like conditional breakpoints, data breakpoints etc.
• The value of the Watch window. I'm still surprised by how much you customize the output to make it easier to find issues.
• The various other windows - threads, memory, etc. etc.
• Etc.

It took a long to discover some of these features. It took even longer to use them almost automatically while debugging.

Obviously the learning curve depends a lot upon the tool you use. Visual Studio tries to be more intuitive and easy to use than something like WinDbg, which is a command-line tool. Even with the ease of use of the visual debugger, you still need to know the typical debugging pattern (using breakpoints) before you can use the tool effectively.

Fewer people have used code profilers than debuggers and the tools are still less mature than their debugger equivalents, so it is harder for new programmers to profile their code than to debug it. In an ideal world we might have a 'fix my code' button or at the very least a 'highlight problem code lines' feature, but for now we need to develop patterns that developers can use to do this themselves.

What features would make profiling easier for you? Are we missing a fundamental concept (the equivalent of 'set breakpoint' in debugging land) that would make things so much easier?