Towards an Improved Definition of Software Quality

This is just a short rant to mark what I intend to become a full essay on the definition of software quality. First there is this website (http://www.improvingsoftwarequality.com/2011/10/qtp-tfs-generic-test-integration.html) that I found in a Facebook post from http://www.facebook.com/pages/Software-Engineering-Microsoft/174748642567856. The page is crowing about the integration of QTP into MS VS. What raises my blood pressure is the casual way in which this page and many other references to software quality seem to raise it as some totem, that if they just say it improves software quality that everyone will believe them. That is just such an overblown and fallacious suggestion that I feel it borders on fraud. So please let me explain why I think this does the software engineering profession no good.

First, software quality is not a single property of software but at best some weighted sum of many constituent properties. The Japanese automakers developed this very nicely decades ago and it has been well developed for software by Carnegie Mellon's SEI. I really don't understand why I continue to hear people refer to it in the singular when it is more appropriate to speak of qualities rather than the singular quality. To use it in the singular with no qualification is to signal a disregard for any form of empiricism since this has as much weight as hand waving.

Second, quality is often meant to be something that is not purely of the software artifact but the software artifact embedded in a given or assumed context. Take performance as an example. Clearly a piece of software that does not perform acceptably on a weak platform may perform adequately on another. It would be wrong to speak of the software as possessing the quality of performance when that attribute only exists for the software/hardware combination.

It might be argued that a property of modifiability is purely a software property since no hardware is required to change the pure software absent its embodiment on a hardware platform. But even here there is an implied context. Modifiability by whom? in what way? against what standard? What is intended in most cases is that the software be successfully changed by an assumed skill level in an understood organization within some unstated unit of time with some unknown defect level as measured over some indeterminate period of time following its installation. That's a lot of unknown information and it is clearly context dependent since the same piece of software could be judged modifiable in one environment and unmodifiable in another. So when it is alleged that the integration of a software testing tool into an IDE will improve quality simply because it makes it more efficient to test the product, I am not convinced.

The ultimate irritant in this fluff piece is the clearly fallacious argument is that the improved environment will have any effect on the program quality at all. The quality improvement comes from finding the ways in which the behavior of the product does not match the specification for that product. Making the testing process more efficient certainly improves the software economics. But that is more an attribute of the software development process and not of the artifact itself. One must assume that to improve the quality of the artifact that the engineer will improve the testing. To assume that testing quality improves because of the time saved due to better integration of the tool set is clearly wrong.

Software quality is clearly a term that must be more adequately defined if it is to be used in any context within ESE. I will begin research finding who has attempted to define this besides SEI and consider expanding this rant into an essay.