Making an Average Decision

Did you know that almost everyone in Sweden has more than the average number of legs? According to Professor Hans Rosling of Sweden's Karolinska Institute, this must be the case because a few people have one or no legs, while nobody has more than two. He uses this fact to illustrate just how silly it is trying to apply the law of averages to many common scenarios.

I've always been fascinated by mathematics. Even though the capabilities with calculus and matrix theory have now percolated out of my aging brain in response to the continual onslaught of new computing technologies and programming languages I encounter in my day job, I just can't resist watching all those TV documentaries about mathematics and (in the case of the latest one) statistics.

Professor Rosling has a wonderful knack of making statistics sound really exciting. He also tosses in some nice anecdotes, such as this quote he heard from an attendee at a lecture: "If unemployment is up by six percent, debt is increasing by a quarter every year, and one in five companies are having difficulty paying suppliers, why are we wasting so much money on compiling statistics?"

It struck me as interesting because I've recently been trying to decide whether to kill off one of my seriously underperforming local community oriented websites, which seems to get more hits from malicious attackers than from real visitors interested in the content. Another example, I suppose, of trying to reduce my attack surface.

It should be really easy to make a decision based on the patterns of traffic to the site. According to an analysis of the last three months, the site gets around 50 unique visitors per day, each viewing two or three pages. However, the site was offline for over a week during the infrastructure failures I suffered in May, so the average is badly skewed both by that lack of availability and the loss of subsequent hits from visitors who gave up trying to access the site and never came back again.

The analysis also suggests that, on average, every third visitor accessed the "recover password" page, and half of the visitors accessed the "register for an account" page. And one on seven visitors was actually a search engine. So, assuming that search engines will spider the complete site, it means that the average number of page hits from real people per visit must be about 0.3. Maybe they only wanted to see the rather nice page header graphic and menu bar? Though the stats also say that, on average, only one visitor in five actually downloaded the page header image - a good indication that even though the site is very well indexed by search engines, there are hardly any real people visiting.

Perhaps the only way to make a sensible decision is to see what the income from the site is compared to the running costs. In my case the running costs are pretty much zero (it runs on a virtualized web server that hosts all my other sites). Though, as there are no fees or advertising revenue either, the income is also zero. So calculating the average ROI per month is not going to be much help.

So, even though the site is supposedly really useful for local people, it's clear that the average of 50 visits per day and two or three page views per visit is nowhere near the real truth. While statistical averages suggest it's doing OK (in terms of its target market), the underlying facts reveal how wrong such simple average numbers really are. It's time, I think, to pull the plug...