usability versus usage -- an iPhone example

Today, I noticed an article in Information Week titled Businesspeople Face Steep Learning Curve with iPhone, which discusses a usability test conducted against the iPhone from a company called User Centric, Inc. This is excellent timing. I've had a post banging around the inside of my head about the difference between usability and usage, and here's a great example of the difference.

Jakob Nielsen says that usability is defined by the following five quality components:

• learnability: how easy it is to do a task the first time you encounter it
• efficiency: how quickly you can perform the task
• errors: how many errors you make, the severity of the errors, and how you recover
• memorability: how well you remember the task and can apply that knowledge in the future
• satisfaction: how happy you are with your experience

These five quality components are measured in most standard usability tests, with a heavy focus on the first three components. You can get a snapshot of the latter two, but this snapshot doesn't necessarily measure them in a meaningful way. Remembering how to do a task ten minutes after I've done it the first time is very different than having to remember how to do a task ten weeks after I've done it the first time. Likewise, my level of satisfaction changes over time. I could have been absolutely over the moon about a feature when I first encountered it, but somehow grew annoyed by it or found that it wasn't as useful a few weeks later.

That is the difference between usability and usage. Usability is measured up-front, and is often a first impression. Usage, on the other hand, is measured over time. Both have their place. If you can't use a feature right out-of-the-box, it's unlikely that you're ever going to get over that initial hurdle.

This iPhone study is a great example of the difference between the two. It specifically compares the usability of an iPhone with the usage of existing phones (both standard 12-button cell phones and smartphones with a QWERTY keyboard). They recruited for users who text frequently on their existing phone, and asked them to do the same tasks on an iPhone and on their own phone. It is entirely unsurprising that users couldn't text on the iPhone as well as they could on their own phones. After all, they know their own phone quite well, and they're frequent texters. Using something entirely new, which has a completely different model of use, will obviously present them with problems. They had expectations based on their current usage of their existing devices, and those expectations don't match up with the iPhone.

These types of studies are pretty common in usability circles. Many competitive usability studies are built around this model: take users of an existing piece of software, and give them tasks against both that software and a competing piece of software that they've never used before. It's a great way to collect feedback about what people like and don't like about both pieces of software. It also helps you to identify places where you might need to overcome the initial usability issues inherent in transferring from one technology to another, and where you'll have to do something to either make the learning curve flatter or to make the learning curve seem like less of an obstacle.

The important thing to remember about this kind of comparison of usage and usability is that it's only a first impression of whatever you're doing the usability test of. In many cases, it is the first stepping stone to doing more work and developing a greater understanding of the differences between two products. User Centric, Inc, says in their iPhone study FAQ that they plan to do further work about iPhone usage. To quote from the FAQ, '[w]e may find that iPhone users with experience may be more efficient, but we need to do the study first'.

I'll be interested to see future studies of iPhone use. I'd love to know if an experienced iPhone user is still slower at texting than a similarly experienced smartphone user, not to mention how satisfied each of them are with their experience. This is the first step towards understanding the difference between iPhone usability and iPhone usage, but it is by no means the last one.

Comments

• Anonymous
  August 17, 2007
  The comment has been removed

• Anonymous
  August 17, 2007
  It's kind of like the difference between measurement precision and accuracy.  They may seem similar but are two quite different concepts.

• Anonymous
  August 17, 2007
  The comment has been removed

• Anonymous
  August 17, 2007
  The comment has been removed

• Anonymous
  August 17, 2007
  Designing a competitive study isn't trivial.  Left to my own devices and given a big enough budget and time frame to work in, I tend to run at least three types of users through them: users of my product, users of the competing product, and users who haven't used either of 'em but would have some reason to.  The third category is obviously difficult to find, but it's not impossible.  Comparing their outcomes is interesting, both in terms of the measurable outcomes (say, time on task or number of errors) and in terms of their perceived outcomes (satisfaction with each application).  Saying "my users prefer my product over the other one" isn't really meaningful, and it's something that I try to avoid.  Barring a lack of alternatives, I already know that they prefer Product A over Product B because they're using Product A.  I don't need to waste a few grand doing a usability study to figure that out.  

• Anonymous
  August 20, 2007
  What is the ratio of prioriies between designing for use and designing for usability? Is it a hard and fast measure? Given that there's always limited time and development resources, and that the likelihood is to produce one solution to a problem rather than a host of them to suit different consumers appropriately, how much focus do the "primary" solutions have on usability? I am mildly concerned that, based on research or not, we consider our consumer base to be inflexible -- that they are deterred by even mild hurdles to use, even when the efficiency of a given use model is higher than the previous use model(s). That we then go on to codify our design principles to eschew innovation that doesn't come in a handy package that gives a seemless transition from Old Way to New Way, thus hiding some efficient models that, perhaps long term, would be more beneficial. -nh

• Anonymous
  August 20, 2007
  Excellent comments! I thought you might find this link valuable to the the discussion: http://www.usercentric.com/UC/news.asp?ID=383 It is the summary of User Centric's first iPhone study on what happened after one week owning the phone... It might give more insight on the usage question as participants also texting tasks and compared their experience to their old phone.

• Anonymous
  August 20, 2007
  Nathan - It's absolutely not a hard and fast measure.  If only life were that easy.  It's all about the priorities at the time.  Those priorities often change from release to release, and they sometimes change mid-cycle as well due to various factors. I think that Excel is a great example of aiming more for usage than usability.  You launch Excel, and you're presented with a huge grid.  Some users have a very evisceral reaction to that grid.  John's comment above about feeling like he's discovered fire by getting a good-looking spreadsheet out of it is just one example of that kind of reaction.  But Excel is a powerful app, and if you put in some time to learn it, it will do magic for you.  There are all sorts of arguments to make about whether we should have had different priorities, but that's outside the scope of this comment. :) I think that WinOffice showed a lot of guts in moving from their old UI to a completely-brand-new UI in Office 2007.  Leaving aside the question of whether the Ribbon was the right thing to do, they really did start from the ground up and allowed a level of UI flexibility that I don't think had ever been considered before.  I think that a lot of people (both inside and outside Microsoft) could learn from that example.  I don't think that it's always necessary to completely start from scratch, but it's a great exercise to entertain the notion and see where it leads you.  It's also a really hard exercise, especially if you're deeply intimate with the software in question.

• Anonymous
  August 21, 2007
  The comment has been removed

• Anonymous
  September 09, 2007
  I read the FAQ, summary and the press release of the testing. Interesting to say the least. Although I tend to think the testing may be flawed in certain ways. Below my comments on this:

1. Multi-tap is widely inefficient, it's no surprise that even a virtual QWERTY board will eventually trump it, especially since iPhone's QWERTY has autocorrection. A better example would be to test 12 button texting using the inbuilt predictive text function and the gap between iphone texting and slow multi-tap would close dramatically. I'm assuming the people they use as tests would understand what predictive text is. Predictive text has been in phones for a decade now and is extremely fast and efficient for 1handed use.
2. The virtual Qwerty is competing with the hard-button qwerty. 12 button use exists by itself and serves it's own important uses/functions. I'm not sure that comparing the iphone qwerty with 12 button usage was the right thing to do, if for no other reason that with predictive text on, the 12 button functions extremely efficiently using only 1 hand. It's a big distinction and really does put the 12 button phones in a class by itself. From the test so far, the information I glean from it is that hard-button qwerty is twice as efficient (and possibly more accurate) than 12button phones, resulting in that even when using 2 hands, iphone keyboard is not signifigantly faster than multi-tap, the slowest form of texting (again predicitive text on 12 buttons is very much faster than multi-tapping). This validates some people's opinion that virtual keyboards are limited. It certainly validates themajorhandset maker's decisions to produce either 12 button or hard-button qwerty. Also validating in some ways, the overall inefficiency of touchscreens in a mobile device that is best served used in 1hand operations.
3. Again I bring up the importance that was missed by the testers. 1hand vs 2hand use. With 1handed use comparable with iphone's 2handed virtual qwerty use, I would say that usability suffers having to utilize 2hands for an operation that could just as effectively be done with 1hand. Compared to a hard button qwerty device, which is built specifically for 2hand use for data entry and 1hand use in most other situations, it mystifies me why the iphone would go full out 2handed operations for very little to zero gain over the 12 button and/or hard button qwerty. In terms of usability, especially when taken into consideration the mobile market and the manner & purpose in which users use their devices, a full 2handed operation doesn't make much sense. It's just reminding me of technology being implemented for technology sake when all the data and evidence was there to bein with that proves that people prefer to use their devices 1handed when going mobile. This limits iphone's UI to a very small segment of the mobile market and apart from being able to easily carry the iphone around, it's not truly a mobile phone that can be used while on the move, carrying a briefcase/books etc. The user necessarily must stop, drop and 2hand use. While I understand that iphone can be used 1handed in certain instances, it's by far not the way it was designed and is not efficient or well working in this respect. In this way, I'm not so sure that as a UI for the mobile market, that it actually improves anything. Giving a good experience is not the same as making things easier to use. Usability must necessarily lead to use. Not just use, but use in the area of operation that it is deployed in. As version 1.0, Iphone UI serves the "experiential" well but lacks serious focus in very important key areas when dealing with this mobile market. Thanks for the links...it's really got me thinking a great deal about this. Cheers!