Inside the usability lab

We're running the v2 "new protection group" wizard through usability testing this week, and I observed a session yesterday. These are fascinating to watch. If you've never had an opportunity to participate in or observe usability testing, here's what ours was like...

The lab is just large enough for table, chair, and computer, and for 2 to 3 people to stand and talk. When the door is closed, it's a quiet room, no distractions. Adjoining the lab is the room I'll call the observatory, in which the tester and others can watch the participant through one-way glass.

The participant (I'll call him Bob) was already settled in at his computer in the lab. I joined the usability expert, Tracy, and Amit, the user interface designer, in the observatory.

Bob had been given written goals for a task, and Tracy had instructed him to explain his thoughts and actions as he proceeded. Now, this "thinking aloud" part is really critical. We don't want to interpret why the user is just looking at the screen while moving his mouse in circles; we want to hear "I'm trying to find a button that will let me start the task" or "I'm reading the screen, it's telling me that I don't have the resources to proceed".

Bob was excellent at thinking aloud. Or perhaps Tracy was excellent at getting the best out of participants? Either way, good results. So, we watched Bob's monitor on a monitor in the observatory, listened as he explained what he was doing and thinking, and took lots of notes.

Occasionally, Tracy prompted Bob to discuss how he would interpret different options on a screen. Bob would answer, "I'm guessing it means..." and his explanation would be correct, but I want to go back and figure out how to make the choices less of a guess.

Bob gave us lots of "I would have expected..." statements; those contain really valuable information. We want to know what the customer expectation would be so we can meet it or manage it.

It was also helpful when he gave suggestions, such as "It wants me to type an email address here; do I need the full address or can it resolve to the address book? I'd like an example on the dialog box."

The difficult part of observing is paying attention, and not the way you might think -- over and over, I wanted to pause Bob while I discussed possible solutions with Amit. Instead, we had to restrain ourselves until the end.

Comments

• Anonymous
  January 01, 2003
  Jeanie Decker and Deb Lewy recently sat in on usability testing for DPM v.2 screens. Why do writers care...