What is Quality?

Most of my career so far has focused on software testing in one form or another.  What is testing if not verifying the quality of the object under test?  But what does the word quality really mean?  It is hard to define quality, but I will argue that a good operating definition is fitness for a function.  In the world of software then, the question test should be answering is whether the software is fit for the function at hand.

The book Zen and the Art of Motorcycle Maintenance tackles the question of what quality is head on.  Unfortunately, it doesn't give a clear answer.  The basic conclusion seems to be that quality is out there and it drives our behavior.  It's a little like Plato's theory of Forms.  This is interesting philosophically, but not practically.  There are some parts which are more pragmatic.  One passage sticks out to me.  As might be suspected from the title, there is some discussion of motorcycle maintenance in the book (but not much).  At one point the Narrator character is on a trip with his friend John Sutherland.  The Narrator has an older bike while John has a fancy new one.  The Narrator understands his bike.  John chooses not to learn about his and needs a mechanic to do anything to it.  Quality then is that relationship between the operator and the bike.  The more they understand it and can fix it, the higher quality the relationship.  In other words, the more the person can get out of it without needing to go to someone else, the higher the quality.

Christopher Alexander wrote about architecture, yet he is quite famous in the world of software design.  His books talk about patterns in buildings and spaces and how to apply them to get specific outcomes.  The "Gang of Four" translated his ideas from physical space to the virtual world in their book, Design Patterns.  Alexander is interesting not just in his discussion of patterns, but also of quality.  What makes a design pattern good, in his mind, is its fitness for a purpose.  He says, "The form is the part of the world over which we have control, and which we decide to shape while leaving the rest of the world as it is. The context is that part of the world which puts demands on this form; anything in the world that makes demands of the form is context. Fitness is the relation of mutual acceptability between these two." (Notes on the Synthesis of Form)

Both authors are making an argument that quality then is not something that can be determined in a vacuum.  One cannot merely look at a device or a piece of software and make an assessment of quality.  In the motorcycle case, the fancy bike would probably look to be of higher quality.  It was the relationship with the owner that made the chopper of higher quality.  With software, it is similar.  One must understand the users and the use model before a determination of quality can be made.

Let's look at a few examples.  Think of the iPhone.  It has a simple interface.  While it has gained complexity over time (it's nearly on version 8!), it is still quite limited compared to a traditional computer.  It has constrained input options, preferring only touch.  The buttons are big and the screens not dense.  Because of this, the apps tend to be simple and single-purpose.  There is no multitasking to be found.  It didn't even have cut and paste when it appeared on the scene.  Yet the iPhone is considered to be high quality.  Its audience doesn't expect to be doing word processing on it.  They want to check e-mail, "do" Facebook, and play games.  It is exquisitely suited for this purpose.

At the other extreme, consider a workstation running Autocad.  Autocad has thousands of functions, many windows open at once, and requires extreme amounts of processing power and memory.  It's user interface is quite cluttered compared to that of most iPhone apps and it is not easily approachable by mere mortals.  Yet it too is considered high quality.  Its users expect power over everything else.  They need the ability to render in 3D and model physics.  It serves this purpose better than anything else in the market.  The simplicity and prettiness of the iPhone interface limits utility and is unwanted in this domain.  The domain of CAD is one of capability over beauty and efficiency over discoverability.

Too often in the world of software we ignore this synthesis of user and device.  Instead we focus on correctness.  The quality of the software is judged based on how correctly it implement a spec.  This is an easier definition to interpret.  It is more precise.  There is a right and a wrong answer.  Either the software matches the spec or it does not.  With a fitness definition, things are more murky.  Who is the arbiter of good fit?  How bad does it need to be before it is a bug?  It can be alluring to follow the sirens of precision, but that comes at a cost.  I will talk about that cost next time.

Comments

• Anonymous
  May 30, 2014
  > Who is the arbiter of good fit? There are two answers to this which I tend to waffle between. -- Pragmatic -- The customer's money. A quality measurement is useful only as a predictor of a product's success or failure in the marketplace. -- Artistic -- The customer's emotional response. A quality measurement is useful as a predictor of how well the product integrates into the consumer's use pattern.

• Anonymous
  May 30, 2014
  The comment has been removed

• Anonymous
  June 04, 2014
  The comment has been removed

• Anonymous
  June 05, 2014
  "The quality of the software is judged based on how correctly it implement a spec. "  I really like this definition, however, I think its a bit incomplete. You can have 2 different software systems implement a spec correctly, but one requiring more coding and unnecessary complexity (introducing more coupling).  I wouldn't say this one produces same quality as the other one.  I think Quality goes hand in hand with Simplicity (which is derived by transforming a complex process into simpler, more modular and scalable parts). So the definition I prefer is "The quality of the software is judged based on how correctly and easily it can implement a spec" ...

• Anonymous
  June 06, 2014
  @Saurab, I don't actual advocate that definition of quality.  I think the idea that quality is related to how well software follows a spec is dangerous.  It assumes a perfect spec, but those rarely exist.  Look for a post on Monday that goes into this in detail. You are correct that there is an aspect of quality that involves how well the code works.  I think this is just a special case of the fitness for function description.  There are two customers for software.  There is the external customer, which experiences it through the interface.  There is also the developer who experiences it through the code.  In the best world, we need to optimize for both types of customers