Thoughts on the Craft of Programming: Abstraction, Refactoring, and How Changes Introduce Bugs - Lesson
(Page 6 of 6 )
The first thing that strikes me is how easy it is to write programs that are almost right, and how hard it is to write ones that are exactly right. The original bug crept in because of some unfortunate choices for C syntax made by the designers of the language and perpetuated by the standards committees (for example, the overloading of the break statement). But even apart from the specific error, the whole style of the original program is really bad12. However, it's quite possible to get large programs with equally poor style to work, more or less. I know because I have seen them and even, I am ashamed to say, written them (before I knew any better). It is possible to get away with programming in poor style because, unlike with physical artifacts, the cost of destructive testing of a program is very small. Therefore it is economically feasible to build a program by an iterative process of construction and testing ("cut and try") in a way that would be out of the question for, say, an automobile or an aircraft. Just imagine what software development would be like if any processor exception caused the CPU to be destroyed.
Unfortunately, it is quite infeasible to perform an exhaustive test of even a small program, and so any program developed by cut and try is bound to contain errors, maybe even serious ones. It's perfectly reasonable to argue that because destructive testing of software is very cheap, the development process should take advantage of this by increasing the proportion of effort that goes into testing and decreasing the proportion that goes into design; but the current sad state of software development is due in large part to driving this argument way past its logical conclusion to an irrational extreme.
To explain this, I'll exaggerate only a trifle: The predominant mode of software development continues to be to slap together a mess of hastily written code, and expect to get it to work by performing tests to find errors and correcting those errors by ad hoc changes to the code. This mode of development is usually accompanied by a management process that anticipates that the defect fixing phase will comprise a very significant portion of the total schedule.
Such a management style typically 1) uses defect counts -- and the rate at which defects are fixed -- as the predominant, if not the only, metrics of progress; and 2) lumps all defects of whatever nature into one defect tracking system, thus making them all somehow comparable. Testing is necessary and even reasonably effective, but our goal should be to develop code in such a way that we do not introduce errors in the first place.13
When we are talking about component-based development and modularity, what we are really talking about is abstraction. What I see in the switch statement in the original piece of code is a total lack of abstraction. To me, the test of whether a given character is a vowel is a simple test of set membership, but I see no reflection of that insight in the original code, and I am forced to conclude that the programmer did not conceive of it in that way at all. Yet it's difficult to see how anyone can make effective use of component-based development without thinking in terms of programming abstractions.
Undoubtedly, part of the problem is the programming languages that are used. C is a fine language in its way, but its direct support for abstraction is limited to functions, macros, and arrays. Pascal provides sets (though they are limited in cardinality), and it would be entirely natural for a Pascal programmer to write is_vowel as "c in ['a', 'e', 'i', 'o', 'u']"; thus a Pascal programmer is more used to thinking in terms of set operations, which are inherently more abstract than tests and assignments. My use of strchr is, of course, nothing but a C implementation of a set membership test, but how many C programmers would see it that way?
It seems to me that we must encourage developers to think in a more abstract way about the programming task. I suspect, however, that this would be an uphill battle. Languages such as APL, Lisp, or Prolog that encourage or require a more abstract view of programming have never been widely popular and are usually regarded as hard to learn. On the other hand, Python, which nicely combines aspects of procedural, object-oriented, and functional languages, has proved quite popular, so there are some grounds for optimism.
In my series of transformations on the original program, I was guided, as much as anything, by an essentially aesthetic sense, most clearly in my feeling that there was something wrong about the nonterminating for loop, and my description of the if statement as "ugly." There is a classic quote from the theoretical physicist Paul Dirac:
"It seems that if one is working from the point of view of getting beauty in one's equations, and if one has really a sound insight, one is on a sure line of progress."
Programs are very rarely beautiful in the Dirac sense14, and we are usually limited to trying to get the ugliness out, rather than the beauty in, but the essence of Dirac's point still applies. We need to instill in developers an intuitive sense of what makes a program beautiful or ugly.
The final lesson I draw is that it is dangerous to make changes to code. It is so easy to unwittingly violate essential preconditions in the existing code, which are usually only implicit and often not completely known even by the original programmers. Furthermore, such violations are often not revealed by even quite thorough testing. The only other known countermeasures, which are not wholly effective, are extreme care on the part of the developers making changes, and thorough review, preferably by people intimately familiar with the existing code.
Notes 1 K&R style: This denotes the style of coding used by Kernighan and Ritchie in their now classic book The C Programming Language, Prentice Hall, Inc., 1988, particularly their placement of braces. Tony Hoare: More properly Sir Anthony Hoare. Perhaps best known for his invention of the Quicksort algorithm, he is now an emeritus professor of computing at Oxford University, having made many important contributions to the theory of programming over the course of his long and distinguished career.
2 "lint" is a venerable Unix utility program that finds errors by static examination of source code. Its name is derived, in the typical Unix fashion that some find irritating, from the notion of removing the "fluff" from code.
3 Because continueis a sort of "do nothing" statement, rather similar to an if-else statement with a null if part; and, of course, continue, like break, is a clandestine goto. In production code I occasionally find a need for break, but I have never used continue.
4 I will readily agree that this is an extremely tiny example, but the string functions of the C run-time library really do form a component, albeit within the strictly procedural paradigm.
5 Admittedly, this is a rather farfetched and contrived example, but it does illustrate the principle.
6 An 8-bit character has 256 possible values, 128 of which are valid ASCII characters. However, C often promotes a char to an int, and in fact will do so in the course of passing the char argument, so the claim that this function may be exhaustively tested is perhaps a little optimistic.
7 We checked with our client and that is really how it is to be defined; if you think that once a pair has been identified both members of the pair should be removed from further consideration that's fine, but it's not what the client wants.
8 In fact, I noticed this bug as I was thinking about the succession of code changes described here, and tested the code only to confirm my deduction of what would happen.
9 I agree that the implementation using strchr is in fact incorrect as it stands, and should be corrected so as not to classify the null character as a vowel. Note that when we originally did the refactoring, count_vowels in fact guaranteed that is_vowel would never be called with the null character as argument, so the implicit precondition could never be violated. Indeed, we did not consider that aspect of the matter at the time. Also, a C programmer is predisposed never to think of the null character as an actual character and so is unlikely to envision null being passed as an argument. That's why oversights of this sort happen.
10 Bertrand Meyer has been the most prolific author on this topic. See, for example, his well-known book Object-Oriented Software Construction and the paper Applying "Design by Contract".
11 The best way to handle exceptional conditions is a large topic in its own right that I do not have space to cover here.
12 Given the context, I am sure the code fragment as presented by Gimpel was abstracted from a larger piece of code and stripped to its bare essentials. I'm not sure whether this would make its style better or worse than that of the original.
13 This hints at the long-running and contentious argument about the utility of formal methods, an argument that's far too complicated to cover here. Close readers may be able to discern where my sympathies lie. Relevant material can be found in:
14 David Parnas, "Why Software Jewels Are Rare," IEEE Computer, February 1996, p. 57. Available (for the moment, at least) online at http://www-inst.eecs.berkeley.edu/~maratb/readings/parnas-jewels.pdf
Originally written by Michael J. Harrison
Copyright (c) 2000-2003 IBM Corporation. All rights reserved.
| DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware. |
