Never convert your story points to time!
When I read these observations on estimations I was reminded of a thing that happened to me a few years ago. The team I worked with started out doing story point estimates and then breaking everything down in the iteration into hours. After the first iteration the team know that X points equaled Y hours. Turned out that X=10Y. When planning the second iteration that knowledge was used to "verify the hour estimates" and sure enough it unveiled a few tasks the team had forgotten during planning. So it all looked like a great way for the team to make sure they didn't miss some important part when breaking down user stories into tasks during sprint planning. At the time for iteration three the story points were no longer used because the team felt it was just an annoying double bookkeeping and everybody knew that one point was ten hours anyway. From that day forward the team consistently under estimated their work. This team did not have the time to establish a track record as the team described in the link above but they also never had a chance I think. Estimating software development in time is very hard and the amount of time you spend on making the estimates does not necessarily improve the estimate with the same factor. Add to that what I wrote earlier this week indicating that the size is more or less useless and only the number of things are important.
But in the defense of time estimates, it is not really the time that is the problem, it is the lack of learning from the past and using historical data to predict the future. I was personally once part of a project involving around 20 developers for one year. Turned out the amount of work it took to complete that project was one man-month less than the initial estimate (i.e. less than 0.5% over estimated). The reason for this? The project manager and key developers had developed virtually the same application for another customer a few years earlier... But that is in my opinion a very expensive way to do estimates so I prefer spending as little time as possible on actually estimating things and more time on understanding the user stories and completing them.