User testing is common sense.

In any industry where members of the public are exposed to a product, rigorous testing takes place prior to its release. It’s necessary. It makes commercial sense. After all, would you go ahead launching a product that customers hated? Who would? But incredibly this happens all the time.

Foregoing user testing is often because of timescales and budgets. It is often excluded from the design process because individuals within marketing or management think they know better. They make critical design decisions based on their own limited knowledge and personal preferences instead of asking the people who matter most - the customers. 

This is a massive mistake. We can’t assume that people think the way we do and we must always remember that we are not our users.  If we test prior to launch then the user experience will be always be improved. In this age of technology there’s simply no justification for not being informed by user testing . It’s a straightforward process, rarely expensive and can often be absorbed within tight timescales without impacting on delivery dates. 

So here’s what you need to know:

 

It’s NOT market research or UAT

User testing is not market research and it is not UAT. It is an entirely different methodology and approach.

Market research activities such as focus groups are designed to find out people’s opinions and attitudes about a product concept. They rely on the user reporting their experiences after consideration and feedback is rarely obtained from an actual product interaction. It is also different from functional testing as this does not involve real users at all. Even UAT, which may simulate typical user journey’s does not involve users and focuses more on system response to actions without consideration of the type of elements that would be explored in user testing.

 

When you should do it

User testing can be carried out at any stage of a design, from the initial concepts to the final product. In general, the most efficient way of carrying out user testing is to do it as early as possible in the design process, so that fundamental problems are eradicated before further design work is carried out. Then test often. Some feedback secured at the beginning of the project is great but it’s not enough. Keep validating your assumptions – even if it involves just showing it to 1-2 users.  The best approach to testing is to start early and test often, using an iterative development process of design, test, update, test – etc.

 

What happens during a testing session

Whether you are conducting a round of testing up to 20 users or a quick validation exercise with 1 to 2, each session should follow the same approach. You want to aim for consistency. If you don’t then you risk undermining the strength of the findings and feedback. During the sessions users will be presented with an early version of “the product”. It doesn’t have to be fully designed – user testing is still valuable if you present just a few screens - but it should give some representation of what the product is about.

Then users will be presented with task-based questions and asked to "think out loud" while they are carrying out a task. What they say reveals their assumptions or beliefs about the product or service, their goals or intentions, and any confusion or misunderstandings. Combined with observations of what the users actually do, this allows us to determine why they are having difficulties, making errors or getting stuck.

The evidence you get is objective and realistic. It is not based on personal opinion and that is the important bit. It will help you understand the users and see things from their perspective.

 

The nitty gritty - costs

Generally, user testing should be conducted by an external specialist agency. They are the independent party, distant from the design and client environment and so therefore able to ensure objectivity in the collected feedback. Costs vary depending on how many people you want to test and the nature of the recruit. If for example, your product is aimed at CEO level individuals, it will be more expensive to secure such participant’s involvement.

Whatever the cost, though the expense of user testing is generally far less than the cost of getting it wrong.

I’ll end this little diatribe with a common statistic, often included by usability agencies as a scary sales tactic. The scariest aspect of course is that this is largely true. Research is well worth the time and money. In the end, it always pays for itself.

“The rule of thumb…is that the cost-benefit ratio for usability is $1:$10-$100. Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design.” (Gilb, 1988)

Comments

• Anonymous
  March 07, 2012
  What do you do if the client won't pay for it?

• Anonymous
  March 07, 2012
  Ideally user testing should be built into the overall UX piece of work but usually it's about education. If the client has already bought into doing UX then it should be easier to convince them of the need but it's a constant battle with some clients. There are various techniques, that don't cost vast amounts of money and can be included into the UX work stream e.g. corridoor testing, which doesn't require you to go out and recruit users and rent labs. Any form of testing is better than no testing and it's finding that balance and creative ways of validating ones work.