Share via

Quality is not Value!

  • Article

I previously blogged about quality and value, but after giving it more thought I determined that quality and value are indirectly related, but quality is not value!

I define testing as any activity designed to evaluate an attribute or capability of a program in order to determine its' ability to meet or exceed implicitly and tacitly applicable standards, guidelines, and/or customer expectations. Even Jerry Weinberg clearly states "Whatever else it means, quality at least means meeting specifications for each job." This does not mean that testing merely verifies functionality against some specification or requirements document. (Only a novice tester would assume testing involves writing tests that simply validate the spec.) But, quality is the specific, measurable, and quantifiable results of the evaluation of the attributes and/or capabilities (especially those that are critical for the success) of a product. Quality attributes and traits can be quantified via various measures, and those metrics should ideally track towards established goals, or compared against baselines or competitors products for example. For example, we can assess quality in terms of some degree of better-ness of comparable attributes for features, or exceeding stated or implied goals or expectations.

Conversely, value is something that serves a useful purpose to some person. When most consumers purchase software, or a vehicle, or a piano they do so because they will, or perceive they will, derive some personal benefit from that purchase. In other words, when something fills a personal need or provides some benefit (either real or perceived), that thing is providing a value to that person. Value cannot be directly qualified; although people can agree or disagree whether or not something provides value to them and they can debate relative degrees of value.

Let's take the example of a vehicle. When I decided to purchase I vehicle I bought a truck because a truck provides a greater value based on my personal needs as compared to a car, a motorcycle, or a SUV. When I started looking for a truck I compared the various attributes and capabilities of the available models in my price range and narrowed it down to a Chevrolet, Ford, and Dodge. Some key attributes I compared were towing capacity, fuel mileage, fuel tank capacity, and warranty. The final decision breaker for me was the crew cab feature with full sized rear doors. For me this provided additional value because when I go skiing I want to be able to take along adult friends (and I don't know about you, but I would be miserable if I were squeezed into an extended cab for a 1 1/2 hour ride up to the ski lodge). So, I went to each dealer and sat in the back seat of each truck and finally settled on the Chevy 1500 HD. For me, a truck provided value and the Chevy 1500 HD met or exceeded the quality expectations I expected in a truck. Now, when people think of a truck they typically don't think of a quality vehicle. Perhaps that is because a truck doesn't provide much value to them, or perhaps they are mentally comparing a truck to a more comfortable sedan which is not a realistic comparison. But, I also think it is quite common for people to confuse quality and value.

Unfortunately, it seems that many managers and testers in the software industry also confuse quality and value. That is why many companies attempt to assess the 'quality' of their products on subjective measures such as smiley sheet surveys that go out to customers, or anti-quality metrics (such as defects found during the SDLC, or defects found post release). Also, some testers are convinced that 'customers define quality.' Poppycock! Customers don't define quality; the project team defines the set of attributes, capabilities or features they think are critical to the success of the project (and will provide some perception of value to the customer), and sets goals that suggest some desired measurable level of acceptance ('quality bar') that meet or exceed expectations in a positive manner for those features and attributes.

Customers who decide to purchase or download software or use services do so because it provides a value for them at reasonable cost (money, time, ease of use, etc.) If the customer likes that software, then to them it is quality software. In this case, the software not only does what they need it to do (value); from their perspective it does it sufficiently well or better than a competitor's product (quality).

Now, I am not providing this perspective simply to be contentious, or insight unreasonable debate and random objections. But, I am beginning to think that one of the primary reasons we can't get our heads wrapped round software metrics and start proposing meaningful solutions to the difficult problem of software quality measurement is because testing is focused on trying to satisfy the end user customer and make the user customer 'happy' rather than providing real, qualitative information to the primary customer of testing whom I consider to be the project's management team. We know the most successful businesses establish quantifiable measurement programs in order to improve in certain areas and reduce costs. But, it sometimes seems that the commercial software industry is content with the status quo, and we trick ourselves into believing we (our company or our product or processes) are getting better based on customer feedback, sales or adoption, and feel-good assessments. Establishing an effective measurement program that establish baselines, and assess improvements in processes, productivity, or cost savings is not easy; it is not quick, it exposes weaknesses or failures in the organization, and it takes time to implement changes and even the measurements must adapt over time. Perhaps we don't need to be more efficient, or increase effectiveness, or reduce costs.  Perhaps our business practices are 'good enough.' In which case, I guess we shall continue to go with the smiley sheets and anti-quality metrics because they are cheap, easy, provide instant-gratification, are non-judgmental, and it doesn't matter if we continue to make the same mistakes over and over again. But, we should acknowledge the fact that smiley faces and anti-metrics do not tell us whether we are improving our processes, or increasing our productivity, or reducing overall (and long term) costs.

Comments