Hard Drive Penalties

In an introductory programming class, I had a professor who explained CPU cache as sitting in a library and having the book in front of you. RAM was like going to a bookshelf. And hitting the hard drive was like ordering it from a store and waiting for it to come.

While those numbers aren't exactly accurate, it does communicate the importance of keeping your data as close to the processor as possible. I recently came across an interesting study by Diomidis Spinellis that shows the relative speeds of different types of memory.

	Nominal	Worst case	Sustained		Productivity
Component	size	latency	throughput	$1 buys	(Bytes read / s / $)
			(MB/s)		Worst case	Best case
L1 D cache	64 KB	1.4ns	19022	10.7 KB	7.91·1012	2.19·1014
L2 cache	512 KB	9.7ns	5519	12.8 KB	1.35·1012	7.61·1013
DDR RAM	256 MB	28.5ns	2541	9.48 MB	3.48·1014	2.65·1016
Hard drive	250 GB	25.6ms	67	2.91 GB	1.22·1011	2.17·1017

 

If you have small chunks of data, the latency difference is huge. L1 cache is roughly 55,000 times faster than the hard drive. If you're dealing with sustained throughput, hitting L1 cache is 283.9 times faster than going to the hard drive.

Draw your own conclusions here, but keep this in mind when you're writing apps. (Note that the cost of memory is a bit outdated in this chart.)