Performance Optimization Philosophy

Optimizing performance is not just about making things run faster, it is about making them run appropriately fast based on perception and cost.

Several questions may be posed at this junction:

• Why does philosophy need to be discussed?
• Isn't performance just about making things run faster?

There is a common misconception that performance optimization is about making things faster.  In actuality, optimizing is actually about finding the correct balance between any number of tradeoffs.  Often and simplistically this optimization is balancing between either cost and hardware (think upgrading processor), or cost and labor (think development/test time).

The other component is user perception.  This is often the most challenging part of the equation and is often what triggers reviews of existing infrastructure due to perceived poor performance.  It is not uncommon that the software is behaving normally and that the issue is not due to a hardware scalability issue, but the end user still feels that the software is not fast enough.  Unfortunately, in these scenarios, there is often little that can be done in the short run without waiting for hardware to catch up with the needs of the code, optimizing the code, or waiting for fundamental architectural changes (like moving from x86 to x64) to come to pass in order be able to eliminate hardware bottlenecks.  In short, the return on investment (ROI) of throwing more hardware at the code to fix user perception may not be there.

Here are some analogies to illustrate the point:
My grandmother is looking for a new car.  She lives on a fixed income and, since she is getting older, doesn't travel much anymore, mostly to the grocery store or bingo, which are within 3 miles of her house.  Due to her needs, the Lamborghini Murciélago is probably a little excessive though it probably performs well.  I'm sure that pretty much any car that runs will be performant enough for her needs.  Furthermore, making the investment in the Murciélago is probably not going to fix her perception that it takes too long to go the 3 miles to the supermarket.

Stretching the analogy a little:
Regardless of what car she gets, it is not a plane, thus it will never ever fly.  But also, there is no guarantee that if she gets a plane it will be faster than a car (the land speed record is 766 mph and a Piper Cub goes about 130 MPH).  Just like it is critical never to assume that moving from x86 to x64 will speed up the application.  It would be more analogous to moving from a car to a tractor trailer where more stuff can be stored (increased addressable memory) so fewer trips (to disk/network) need to be made.