Annotated Bibliography

Code Reading and Program Comprehension

A - H I - P Q - Z

Abstract: Because a system's software architecture strongly influences its ability to support quality attributes such as modifiability, performance, and security, it is important to be able to analyze and reason about that architecture. However, architectural documentation frequently does not exist, and when it does, it is often out of sync with the implemented system. In addition, it is rare that software development begins with a clean slate; systems are almost always constrained by existing legacy code. As a consequence, we need to be able to extract information from existing system implementations and reason architecturally about this information. This paper presents Dali, an open, lightweight workbench that aids an analyst in extracting, manipulating, and interpreting architectural information. By assisting in the reconstruction of architectures from extracted information, Dali helps an analyst redocument architectures and discover the relationship between "as-implemented" and "as-designed" architectures.

Comments D&N: This influential book tries to do for programming what Strunk and White did for writing. The authors want people to read programs and thereby learn to program better. This slim volume is filled with snippets of advice (Make your programs read from top to bottom.) and illustrative code segments from a variety of languages. It is not the ultimate authority some would make it out to be, but it is a stimulating classic that everyone interested in serious programming should read. It is somewhat dated, but contains a lot of good advice.

Comments D&N: In The Elements of Programming Style, Kernighan and Plauger illustrate their points with other people's code. Here, they use their own simple, reusable software tools, written in Pascal. The book is virtually a whole course on programming technique. There is much code to read here, but, as is commonly the case in textbooks, most of it is inscrutable without the surrounding discussion. Thousands of programmers have studied this book on their own. (This is a revision of an earlier book, Software Tools. The programs in that book are written in Ratfor, which requires a preprocessor whose output is FORTRAN code.)

Comments D&N: Knuth describes his WEB system for programming and documentation. Anyone with a deep interest in the system should read this paper, but the reader who would prefer a brief, lucid description of this interesting system (and philosophy) should read Jon Bentley's piece [Bentley86a] on the subject instead.

Source code for Knuth's typeface-generation system, written using WEB. (See [Bentley86a].)

Comments D&N: Source code for Knuth's text-processing system, written using WEB. The book runs to nearly 600 pages. (See [Bentley86a].)

Abstract: Program comprehension is a complex problem solving process. We report on an experiment that studies expert programmers' comprehension behavior in the context of modifying a complex PASCAL program. Our Data suggests that program comprehension is best understood as a goal-oriented, hypotheses-driven problem-solving process. Programmers follow a pragmatic as-needed rather than a systematic strategy, they restrict their understanding to those parts of a program they find relevant for a given task, and they use bottom-up comprehension only for directly relevant code and in cases of missing, insufficient, or failing hypotheses. These findings have important consequences for the design of cognitively adequate computer-aided software engineering tools.

Abstract: There has been some work on the improvement of quality of software documents, especially requirements documents, in the last few years. Major areas of activity range from the development of formal methods to the use of inspections. But there has been little ativity on the topic of reading techniques. We believe this to be the most critical analytic techniques we have for assessing software documents. Reading can be performed on all associated software documents as soon as they are written without any machine support. Reading has been shown to be more effective than testing in uncovering faults. Yet there have been very few techniques developed for reading. We have developed a new reading technique called perspective-based reading for requirements. The idea behind perspective-based reading is that various customers of a documents should read the document from various points of views. This paper describes the technique in detail. To assess the efficiency of the technique a controlled experiment with professionals from NASA and Computer Sciences Corporation was conducted. The participants should read four requirements documents. In the controlled experiment the participants have found more defects in three documents with perspective-based reading than with their own technique. The preparation, execution and analysis of this experiment is the second major part of this paper. It describes the work that was done at the University of Maryland. It also includes some proposals for further studies and research at the University of Kaiserslautern based on the experiences in Maryland. Apart from a replication or variation of the perspective-based experiment the proposals also include a brief introduction into the problems of traceability. This can be seen as a first step in this area which can be extended in future.

Abstract: This paper reports on an empirical study of the cognitive processes involved in program comprehension. Verbal protocols were gathered from professional programmers as they were engaged in a program understanding task. Based on analysis of these protocols, several types of interesting cognitive events were identified. These include asking questions and conjecturing facts about the code. We describe these event types, and use them to derive a computational model of the programmers' mental processes. Letovsky refers to a study involving the videotaping of six professional programmers as they enhanced a FORTRAN 77 program of about 250 lines. (The same study is also the basis for [Letovsky86b] and [Littman86].) Subjects were asked to think aloud as they worked. The author describes and analyzes what they said as they labored to understand the program to be modified. He presents a cognitive model of program understanding composed of the programmer's knowledge base, a mental model, the construction of which is the ultimate goal of program reading, and an assimilation process by which the programmer actually builds the mental model. Most of the paper is concerned with the assimilation process and the empirical data justifying the author's analysis of it.

Comments D&N: Although Letovsky's language often differs from that of Brooks, his cognitive model of program comprehension is basically consistent with and elaborates the model in [Brooks83]. Whereas Brooks emphasizes top-down approaches to reading programs, Letovsky offers convincing evidence that programmers work both top-down and bottom-up. Much of the paper is devoted to analysis of the questions, conjectures, and inquiries made by the programmers while reading the code.

Teacher and student alike can benefit from reading this paper, which suggests, perhaps better than any other, what a useful model of program comprehension might be. Examples from the data contribute to one's understanding of the assimilation (reading) process on one hand, yet detract from the author's description of his model on the other. Practical implications need to be drawn by the reader. Asking students what Letovsky's results imply (about documentation, for example) should evoke interesting discussion. The 1987 reprint includes an appendix, Other Categories of Questions and Conjectures, which illustrates programmer thinking not accounted for by the author's model.

Comments D&N: The authors conclude, based on the same study as [Letovsky86a], that inadequately documented delocalized plans are sometimes responsible for misreading of programs on the part of maintainers. They analyze comprehension failures by their subjects and suggest techniques to prevent such misunderstandings when composing programs.

According to this paper, the task of understanding a program is one of uncovering the intention behind the code. Intentions are described as goals. Techniques for realizing goals in a particular implementation are called plans. Plans are a lot like algorithms, but they may involve non-contiguous elements and may be combined in ways we do not usually consider for algorithms. Two plans involving loops may be combined into a solution using a single loop implementing two distinct goals, for example.

The authors have observed that readers of programs tend to infer the goals of code fragments on the basis of locally available information. If the plan for a fragment is delocalized, that is, part of the plan is realized in non-contiguous code, the reader will often incorrectly perform this inference. The authors suggest various documentation techniques to mitigate reading problems resulting from delocalized plans, most of which require the programmer to be more explicit in comments about his intentions. The paper also includes a brief section on related work and tools for assisting program reading.

The comprehension difficulties discussed here are not surprising ones, yet the paper comes as something of a revelation to most of us who have never thought much about those difficulties or have never thought about them so clearly. This is must reading for student and teacher alike.

Capsule Description: This module, which is directed specifically to software engineers, discusses the writing process in the context of software engineering. Its focus is on the basic problem-solving activities that underlie effective writing, many of which are similar to those underlying software development. The module draws on related work in a number of disciplines, including rhetorical theory, discourse analysis, linguistics, and document design. It suggests techniques for becoming an effective writer and offers criteria for evaluating writing.

This curriculum module is a brief presentation of what software engineers should know about written technical communication. Levine, Pesante, and Dunkle believe that future software engineers must be taught to write well. They provide substantial advice to instructors who may be sympathetic to this idea but who are uncertain of what they can do to implement it. This is an essential resource for teachers who sincerely want their students to write better.

Comments D&N: An extended apology for and explication of structured programming. The book provides a good description and adequate examples of the algorithmic conversion of arbitrary programs into structured ones. Of greatest interest for our purposes is the 66-page Chapter 5, Reading Structured Programs. The entire book is sprinkled with exercises.

Much of Chapter 5 is devoted to an example of how an unstructured, undocumented program can be structured and documented bottom-up using stepwise abstraction, the formulation of an abstract description of what a fragment does from the fragment itself. (The techniques described here provide the basis for what Basili and Mills do, with greater formality, in [Basili82].) The chapter is notable for its insights into program reading generally and into the proper nature of comments. According to the authors, well-documented programs can largely be read top-down, whereas poorly documented programs have to be read mostly bottom-up. In practice, they suggest, both strategies are usually used, even for well-documented or totally mysterious programs.

Teachers should read Chapter 5 and whatever other sections they find of interest. Reading Structured Programs is full of practical ideas useful when teaching program reading. Students who do not need to be convinced of the virtues of structured programming and who do not need to structure spaghetti code are likely to find this book tedious.

Abstract: Understanding how a program is constructed and how it functions are significant components of the task of maintaining or enhancing a computer program. We have analyzed videotaped protocols of experienced programmers as they enhanced a personnel data base program. Our analysis suggests that there are two strategies for program understanding, the systematic strategy and the as-needed strategy. The programmer using the systematic strategy traces data flow through the program in order to understand global program behavior. The programmer using the as-needed strategy focuses on local program behavior in order to localize study of the program. Our empirical data show that there is a strong relationship between using a systematic approach to acquire knowledge about the program and modifying the program successfully. Programmers who used the systematic approach to study the program constructed successful modifications; programmers who used the as-needed approach failed to construct successful modifications. Programmers who used the systematic strategy gathered knowledge about the causal interactions of the program's functional components. Programmers who used the as-needed strategy did not gather such causal knowledge and therefore failed to detect interactions among components of the program.

Comments D&N: This empirical study reports on 10 professional programmers performing the maintenance task described in [Letovsky86a]. (Why the two papers report different numbers of subjects is not explained.) The authors argue that different subjects applied different strategies to the program reading task. The key to successfully modifying the target program was understanding interactions among components-presumably a not uncommon situation-and only subjects who attempted to understand the overall operation of the program were successful. Significantly, years of experience was correlated neither with successful modification nor systematic strategy. The authors admit that the subjects might have behaved differently had they been able to test and debug, and they also admit that it is impractical to try to understand truly large programs completely before attempting modification. Nonetheless, they speculate that effective program reading prior to changing any code leads to more efficient maintenance.

This paper is perhaps most useful as a means of sending a warning to students inclined to begin modifying a program before they understand it.

Comments D&N: Lukey reviews and comments on progress in understanding program comprehension and debugging. The author suggests that there are three methods of studying these phenomena-the experimental approach, the artificial intelligence approach, and the observational approach. He focuses on the former two.

This is a good summary of a large body of research, much of which, because of limitations of space, we have not been able to include here. Although, as a literature review, this material is somewhat out of date, it provides helpful coverage of early references. Lukey's review is recommended for teachers and for students with serious interest in studying program comprehension.

Abstract: The consensus in the programming community is that indentation aids program comprehension, although many studies do not back this up. We tested program comprehension on a Pascal program. Two styles of indentation were used-blocked and nonblocked- in addition to four possible levels of indentation (0, 2, 4, 6 spaces). Both experienced and novice subjects were used. Although blocking style made no difference, the level of indentation had a significant effect on program comprehension. (2-4 spaces had the highest mean score for program comprehension.) We recommend that a moderate level of indentation be used to increase program comprehension and user satisfaction.

This paper addresses the impact of indentation and blocking on program comprehension. By blocked indentation, the authors mean that statements immediately within a begin ... end pair share a common left margin with those delimiting keywords. In nonblocked style, the delimited statements are indented further. The authors review previous studies of indentation, noting that their support for the hypothesis that program indentation aids program readability and comprehension is, at best, ambiguous. They explain possible reasons for the discrepancy between their results and those reported by others. The results of the experiments reported here favor the view that indentation aids comprehension, but they also show that excessive indentation (6 or more spaces) does not increase the effect. Interestingly enough, the novices in the study reacted very favorably to indented code and rejected the nonindented program. Experts, on the other hand, showed no such prejudice. This is a good paper on the effect of formatting practices. The experiment was done carefully, which makes it especially relevant to those interested in empirical studies. The practical wisdom to take away from the paper is simple: indent code three spaces to show its structure. This paper is recommended for both instructors and students.

Comments D&N: A discussion of basic algorithms illustrated with Pascal examples. The book is distinguished by the inclusion of complete, documented programs to accomplish simple tasks. It makes pleasant bedtime reading for any programmer interested in clear, straightforward coding and documentation. Reading exercises are included with the complete programs, and teachers may find these useful. Many questions are too broad and open-ended for students not accustomed to program reading, however.

Abstract: Software engineering research has focused primarily on software construction, neglecting software maintenance and evolution. Observed is a shift in research from synthesis to analysis. The process of reverse engineering is introduced as an aid in program understanding. This process is concerned with the analysis of existing software systems to make them more understandable for maintenance, re-engineering, and evolution purposes. Presented is reverse engineering technology developed as part of the Rigi project. The Rigi approach involves the identification of software artifacts in the subject system and the aggregation of these artifacts to form more abstract system representations. Early industrial experience has shown that software engineers using Rigi can quickly build mental models from the discovered abstractions that are compatible with the mental models formed by the maintainers of the underlying software.

Abstract: Two of the most important and most difficult questions one can ask about a new software development technique are whether the technique is useful and whether the technique is usable. Various flavours of empirical study are available to evaluate these questions, including surveys, case studies, and experiments. These different approaches have been used extensively in a number of domains, including management science and human-computer interaction. A growing number of software engineering researchers are using experimental methods to statistically validate hypotheses about relatively mature software development aids. Less guidance is available for a developer of a new and evolving software development technique who is attempting to determine, within some cost bounds, if the technique shows some usefulness. We faced this challenge when assessing a new programming technique called aspect-oriented programming. To assess the technique, we chose to apply both a case study approach and a series of four experiments because we wanted to understand and characterize the kinds of information that each approach might provide when studying a technique that is in its infancy. Our experiences suggest some avenues for further developing empirical methods aimed at evaluating software engineering questions. For instance, guidelines on how different observational techniques can be used as multiple sources of data would be helpful when planning and conducting a case study. For the experimental situation, more guidance is needed on how to balance the precision of measurement with the realism necessary to investigate programming issues. In this paper, we describe and critique the evaluation methods we employed, and discuss the lessons we have learned. These lessons are applicable to researchers attempting to assess other new programming techniques that are in an early stage of development.

Abstract: Studies of software comprehension often use short-term recall as a way to study comprehension. Experiments range from broad descriptions of program purpose to tests that require subject to 'fill in the gaps'. There is little doubt that human memory is a very important issue when it comes to understanding how software systems work. This paper addresses the topic of 'long term' comprehension, namely, the retention of programming knowledge over a period of months, years or even decades. Long-term retention of programming knowledge is thought to be an area within the psychology of programming literature that has received surprisingly little attention. To stimulate debate, methodological issues that may affect the study of long-term comprehension are detailed. Finally, a planned experiment based upon related psychology of programming studies is outlined.

Comments D&N: A condensed version of [Oman90b], but with a few useful ideas and comments not found in that more scholarly paper. Its brevity makes it attractive for student reading.

Comments D&N: The authors discuss relatively straightforward source code formatting and commenting techniques to improve program comprehension. They also discuss their experimental evidence to support their claim that the techniques do, in fact, achieve their objective.

Following a brief review of the literature, Oman and Cook introduce their book format paradigm as a vehicle for displaying source code. They point out that the organization of books into chapters, sections, and paragraphs, supplemented by prefaces, tables of contents, and indexes, is both familiar and serviceable. Similar organization and devices can be applied to computer programs, and most of the application can be done automatically.

A program formatted according to the book paradigm includes a preface, table of contents, chapter divisions, pagination, and indices. Most of the added text is in the form of comments, although page headings, which include chapter name and page number, are apparently generated by a listing program. In the table of contents, for example, one might find the main procedure listed as Chapter 2, beginning on page 4. A procedure called on page 4 would be listed in a module index at the end of the program as being called from the main procedure; and, if it called other modules, those would also be noted.

The authors refer to features that aid the reader in finding his way around the program as macro-typographic factors. They also discuss micro-typographic factors, including the addition of blank lines and indentation, vertical alignment conventions, use of upper- and lowercase, use of boldface, addition of paragraphing (putting more than one statement on a line), etc. All these conventions were selected by appeal to typographic style principles, which the authors claim is supported by the empirical evidence.

The paper also reports on four experiments carried out with Pascal and C code formatted according to the book paradigm or formatted conventionally. Subjects were asked to implement an enhancement, complete a comprehension test, or complete a call graph for the code in question. Data were also gathered through think-aloud protocols in one of the experiments. In each case, subjects using book format code outperformed their fellow subjects using more conventional program listings. Moreover, the authors report that they adapted easily to the book format, even in the absence of instruction in its use.

Oman and Cook conclude that their book paradigm is natural and useful, though they recommend that students be taught style principles rather than particular formatting conventions. This is an important and thought-provoking paper. It is easy to quibble about the specifics of the book format paradigm, but it is difficult to dismiss the thrust of this work. The authors' recommended conventions are by no means radical, and require no substantive changes in executable statements of the code. Yet, the claimed results are impressive. This is must reading for teachers. Students not specifically interested in empirical studies should probably read [Oman90a] instead.

Abstract: DS-Viewer is a tool that is the result of a research project in data structure presentation within a program state. This tool addresses two distinct issues in this area: (1) to effectively present data structures themselves for a given program state and (2) to present groups of data structures and their interrelationships as described by their pointer definitions. Graphical presentations were developed to address these issues. For the data structure presentation, the user is provided a display window for any single data structure instance formatted with its fields and field values. Flexibility in display is provided by allowing the user a choice from the various value formats for each field. For groups of data structure instances, a graphical drawing space is provided in which pictures of these data structure instances and their interrelationships are drawn as blocks and arrows. The computer assists the user in drawing such a picture by describing its components, allowing the user to choose which to draw and to construct as much of the picture as desired.

This paper describes an IBM prototype system that runs on a PC as a Microsoft Windows application. The tool is designed for use in debugging complex data structures that may themselves have been corrupted. DS-Viewer allows the user to select interactively the data structure components and representations to be used for the graphic presentation. It allows the user to select a display of multiple instances of data structures linked by pointers.

Abstract: This report focuses on differences in comprehension strategies between programmers who attain high and low levels of program comprehension. Comprehension data and program summaries are presented for 40 professional programmers who studied and modified a moderate length program. Illustrations from detailed think-loud protocol analyses are presented for selected subjects who displayed distinctive comprehension strategies. The results show that programmers attaining high levels of comprehension tend to think about both the program world and the domain world to which the program applies while studying the program. We call this a cross-referencing strategy and contrast it with strategies in which programmers focus on program objects and events or on domain objects and events, but not both.

Comments D&N: Based on her previous research, Pennington proposes that understanding of overall program flow control precedes the more detailed understanding of program functions. In particular, she suggests that program readers build at least two mental models of the program they are studying, a program model and a domain model. The program model is characterized by an abstract knowledge of the program's text structures. The domain model relates objects and functions in the problem domain to source-language entities.

The author carried out an experiment using a minimally documented 200-line FORTRAN program. Subjects were asked to study the program for 45 minutes in preparation for a modification task. Some of the subjects were asked to think aloud as they examined the program. After the study period, subjects wrote summaries explaining what the program did and answered 20 comprehension questions. They were given an additional 30 minutes to implement the requested change, after which a second summary was written and 20 more comprehension questions answered. Using her analysis of the data, Pennington asserts that the comprehension strategies of the subjects can be characterized as program-level, domain, or cross-referencing, the latter being a strategy that combines features of the other two. That is, the programmers concentrated either on the program, on the problem domain, or somehow effectively related the two. Not surprisingly, it was the cross-referencing readers who performed best.

Whether or not Pennington's results indicate that program readers create two distinct mental models in succession, they certainly support the layered abstractions proposed by Brooks [Brooks83] and Letovsky [Letovsky86a]. This is an insightful paper discussing the cognitive process of program comprehension. It is equally interesting on methodological grounds. Pennington's paper is recommended reading for instructors interested in program comprehension. Student benefits from reading the paper, however, may be limited.

Abstract: Assertions or more generally "Programming by conract" have gained widespread acceptance in the computer science community as a means for correct program developmnet. However, the literature lacks an empirically evaluation of the benefits a programmer gains by using assertions in his software development. This paper reports about two controlled experiments to close this gap. Both experiments compared "Programming by contract" to the traditional programming style without assertions. The evaluation suggests that assertions tend to decrease the programming effort and that assertions lead to more reliable programs compared to those programs written without using them.

Abstract:

Abstract: There is widespread anecdotal evidence that expert programmers make use of visual mental images when they are designing programs. This evidence is used to justify the use of diagrams and visual programming languages during software design. This paper reports the results of two studies. In the first, expert programmers were directly questioned regarding the nature of their mental representations while they were engaged in a design task. This investigative technique was used with the explicit intention of eliciting introspective reports of mental imagery. In the second, users of a visual programming language responded to a questionnaire in which they were asked about cognitive processes. The resulting transcripts displayed a considerable number of common elements. These suggests that software design shares many characteristics of more concrete design disciplines. The reports from participants in the two studies, together with previous research into imagery use, indicate potential techniques for further investigation of software development support tools and design strategies.

Abstract: Using design patterns is claimed to improve programmer productivity and software quality. Such improvements may manifest both at construction time (in faster and better program design) and at maintenance time (in faster and more accurate program comprehension). This paper focuses on the maintenance context and reports on experimental tests of the following question: Does it help the maintainer if the design patterns in the program code are documented explicitly (using source code comments) compared to a well-commented program without explicit reference to design patterns? Subjects performed maintenance tasks on two programs ranging from 360 to 560 LOC including comments. Both programs contained design patterns. The controlled variable was whether the use of design patterns was documented explicitly or not. The experiments thus tested whether pattern comment lines (PCL) help during maintenance if patterns are relevant and sufficient program comments are already present. It turns out that this question is a challenge for the experimental methodology: A setup leading to relevant results is quite difficult to find. We discuss these issues in detail and suggest a general approach to such situations. The experiment was performed with Java by 74 German graduate students and then repeated with C++ by 22 American undergraduate students. A conservative analysis of the results supports the hypothesis that pattern-relevant maintenance tasks were completed faster or with fewer errors if redundant design pattern information was provided. Redundant means that the information carried in pattern comments is also available in different form in other comments. The contribution of this article is twofold: It provides the first controlled experiment results on design pattern usage and it presents a solution approach to an important class of experiment design problems for experiments regarding documentation.

Sources A - H I - P Q - Z

Comments should be sent to

This document
Created: March 8, 1996
by RLU

Modified: January 11, 2002