Challenges in Test – Knowing your customer
So you know your product & the code, what else is there? Your customer! They are the ones that will be using your product to solve their real problems.
Knowing your customer
So your PM, Dev, & you have implemented the greatest feature. Do you know what problem this is going to solve for your customer? Do you know how they expect to use it? If they are going to be deploying your product do you know what type of topology they will be using?
A good example of this is the Mars Rover that was lost in 1999. One team designed their system using English units, but the other team was actually using Metric units. I bet their tester ensured the feature worked wonderfully but there was one costly assumption that was overlooked & after a 286 day journey the probe was burned up in the atmosphere.
As a test team you will be charged with figuring out how you will be testing the product. The assumptions you make will wind up impacting your customer & their experience with your product.
Developer Bias – “It works on my box”
During the course of development it is easy for something I will call developer bias to set in. You have been working on the product & seen it evolve over time. You can easily grow accustomed to issues or why something might work the way it does even though it is not intuitive to your customer.
One refrain that exemplifies this is a phrase you might hear from a developer of “It works on my box.”, when you report an issue with the product.
Take for example a server product which uses SQL as a datastore. During development most of the team will likely install the product all on a single box. Who wants to maintain multiple machines if they don’t have to after all. But customers won’t be running in that configuration & will likely have a SQL machine & run your server on a separate machine. Does your product work in that topology? Oh & how did you install your SQL server? SQL server can have named instances. Can you install to a named instance? Does your product work when installed to a named instance?
Customer Personas – “It works great for me”
Although you may be one of the target customers for your product, are you a true representative of all your customers? What does a teenager expect form your product vs. a stay-at home mom? Each customer has their own unique needs & scenarios that they are interested in, you need to be aware of them and how you can cover them in your testing.
Let’s look at something such as Kinect for Xbox 360 & a feature as simple as someone waving to launch the Kinect Hub. Seems pretty straight forward right? Interestingly enough eHow even has instructions that are as simple as 1, 2, 3.
- Lift your hand toward the person to whom the wave is being directed.
- Spread the thumb away from the other fingers of the hand and hold the rest of them together.
- Pivot at the elbow & move your hand from side to side several times
It all seems so simple, but do a sampling of people & have them wave at you. You will find a large variance in how they actually wave to you. Some might just slightly move their wrist, others might move their hand back & forth very rapidly. Are kids part of your customer set? They can wave even more differently with their hand almost sideways moving more up & down.
Conclusion
But I am just a tester, isn’t PM the one that talks to customers? They do, but do they understand the assumptions you are making in testing? Have you used the many resources you have to find out information from your customers? At Microsoft there are several avenues to get this information: TechNet Forums, Pre release programs through connect.microsoft.com, various conferences, external user groups, internal distribution lists, MVPs, etc. Work with your team to reach out to your customers, I am sure they will be more than happy to help you build a great product that will help solve their problems.