25 Hopper hours in the real world – how long will my device last?

A very popular Hopper question I get is “How can I calculate actual expected device life from the 25 hour Hopper requirement?”

 

Answer: you can’t – there is no way to calculate or ‘generalize’ how long your device/application will survive in real-life with the just the Hopper number. Hopper was designed at its foundation to measure code stability – not scenario stability.

There is however a direct correlation between Hopper-healthy builds and lack of bugs reported by our in-house dogfooding effort. Conversely, pre-release builds that did not survive Hopper were a constant reliability pain to our users. Hopper numbers have the added benefit of being a good reference point that can be useful in relative comparisons like “did my recent changes affect the stability of the device”? It must be said that Hopper is only one measure of device stability and the following are considerations each OEM must take into account when designing their reliability test plans:

First , Hopper is completely random and most of its time is spent doing random things. Your users (hopefully) will be spending most of their time doing non-random things and a very small percentage acting like Hopper.

 

Secondly, Hopper is not required to run on a “provisioned” device. That is, we don’t require a configured data connection, configured inbox accounts or “real” data that would affect code paths or decisions based on state. There is nothing preventing your ambitious development team from doing so, it is just that we don’t require it.

 

Lastly, Hopper doesn’t do any kind of validation such as “Did I actually delete that contact”? Hopper doesn’t know deleting contacts from playing media files – Hopper only cares that the device is still responsive and capable of launching new apps (but doesn’t actually verify the desired application was the one that was launched).

 

Does the above explanation diminish the value of Hopper or the bugs it finds? No, not even a little bit. My explanation is simply meant to expose opportunity for OEM expansion and better understanding of what is required to build stable and reliable devices. The key take-away from this blog should be that Hopper is only a single measurement of stability, not the only measurement.

Comments

• Anonymous
  August 31, 2006
  thks for your effort, it's Clear enough.

• Anonymous
  March 08, 2007
  Can hopper be run on Smartphone

• Anonymous
  July 07, 2007
  in fact ,i dont think hopper is as useful as you mentioned.