Richard Harter’s World
Site map
March 2008

Production quality software

Once upon a time, some twenty years ago, I wrote the article that follows. At that time “open source” was not a buzzward. In the afterwords section I add a few words on how much things have changed.

People have different images of production quality software. Here is mine. Significant features of production quality software include:

    (li) The software will run by a number of people (customers) who have paid hard cash for the software. These people will not know the details of the software and usually will not have access to the source. It must be possible for these people to run the software without a knowledge of its contents and without contact with the creators of the software.
  1. Pursuant to (a) the software must be substantially and correctly documented from the prospective user’s viewpoint. This documentation should not presume special knowledge available to the creator of the program.
  2. The software should be substantially error free (ideally it should be completely error free — however.) It should work not only when used by the creator, but also when used by the end users. In particular there should be no “don’t do that because we didn’t take care of that case” loose ends.
  3. The software should be rigorously tested. It is not enough that it works in the small number of situations of immediate interest to the creator. It must work under a wide variety of situations. Testing and validation is a major part of the work in preparing production software (or ought to be.)
  4. The software should have detailed error handling, and input and data validation. Error recovery and proper handling of erroneous input should comprise a significant portion of both the code and the design.
  5. There should be either implicitly or explictly a quality control process.
  6. There should be carefully controlled procedures for generating the software actually shipped to the customers. All released software should be clearly identified as to content and origin.
  7. Typically, production software is the product of more than one person and may run from tens of thousands to millions of lines of code. The entirety of the software should be coherent in design, methodology, and style, and should be readily maintainable as a whole. The work of individual contributors should fit into the main body.
  8. The software should be well engineered. This includes analysis of a variety of tradeoffs.

I could go on. Perhaps I should make the caveat that much software that is commercially available is not of production quality by the standards I have listed :-). However I wouldn’t list fancy help screens as a criterion for production quality software. A nicety, perhaps, but not a gut issue.

Now let us consider a typical program in an academic setting. [That is, if there is such an animal — BSD4.x and the beginning student’s first program don’t have much in common.] I suggest that the following features are common:

  1. The program is used principally by the creator and a few close associates.
  2. The program is freely modifiable by the users if it does not work to expectations.
  3. Testing is principally restricted to the immediate uses intended by the creator.
  4. The software is not part of a much larger body of software.
  5. Version control and release control are not major issues.
  6. Long term maintenance by people other than the creator is not a major issue.
Note that I am not claiming that these features are always the case, but merely that they are very common. Nor am I claiming that programs of these kinds are shabby or somehow undesirable. There are things that should be done in one context and are irrelevant in another. A good Software Engineer should know what these things are. If I am writing a program which will be used by 10,000 people there are things that I had better do that are not important if I am going to be the only user.

An academic instructor typically teaches his students to write programs that are well written but are not elaborated. goes on to say that he feels that he (and presumably his students) Instructors often that their time and their students tims his is more profitably spent in learning. I agree. Clearly, one should not invest more effort into a program than is warranted by its ultimate value (as measured by the total usage of the program.) This is the kind of engineering tradeoff decision that one makes all of the time.

Nonetheless there are substantive differences between software production in an academic setting and in “industry” as I hope I have made clear. And I will say, dogmatically, that it rests on academia to understand what those differences are and to teach something about the nature of producing production quality software.

I would also add that there is a common fault in new programmers, which is, essentially, they have an underdeveloped sense of responsibility and professional obligation (sometimes known as professional ethics). [It is not infrequent in old programmers also.] In part, this is because we all like to play with the machines. In part it is simply the case that young people are often still making the transition from play to work. The idea that one owes one’s principal loyalties and efforts to one’s employer and that one’s activities should not impede your fellow employees is one that has to be learned [quite often rather rudely from your immediate supervisor.] It may well not be legitimate to blame the universities for this state of affairs; however I will express the wish that professional ethics and a minimal knowledge of what will be expected of them as employees were a part of the core curriculum.


In some respects there have been major changes in the software engineering profession in the past twenty years.

This page was last updated March 21, 2008.

Richard Harter’s World
Site map
March 2008