Program Comprehension

I. Introduction

We tend to focus on constructive activities in the early computer science courses. The explicit goal is to have the students producing programs written in some high-level language a quickly as possible. The justification for such behavior is fairly straightforward, student must demonstrate their competence in language constructs, and program writing provides the opportunity for that demonstration. Hence the focus in early courses is language acquisition, and the use of language in solving a variety of problems.

At the level of language constructs, a student's introduction to code reading is problematic. Examples are provided to show the way to use various constructs. Students are expected be read the example as an exemplar, the focus, and hence the student's attention, is on the construct, not on understanding how such an assembly of statements solves a particular piece of the problem. This instructional regime in combination with the constructive approach mentioned in the previous paragraph leaves program comprehension an accidental outcome of computer science education. Yet the available evidence suggests that our current neglect of the topic cannot be justified by the argument that adequate program reading skills develop naturally and without special encouragement in students otherwise well prepared to enter professional practice. Deimel90, p. 6

The lack of attention to program comprehension skills is somewhat puzzling. Code reading, program understanding, and program comprehension appear to be essential for the learning of computer science and programming, as well as, the practicalities of the practitioner. Certainly understanding an algorithm requires considerable effort. If the practitioner/student cannot fully comprehend the material presented then they are doomed to invent their own strategy. An agreeably less than optimum situation.

The world of software development abounds with opportunities to engage coded passages in an attempt to understand the material, or the intent of the writer. The software maintainer must comprehend a given program sufficiently well to plan, design, and implement modifications. Though there may or may not be adequate supporting documentation, the programmer is bound inextricably to the code to determine how the current program performs its tasks. It would appear from this that ones ability to "make sense" of this legacy artifact (estimates 47-62% of maintenance activity relates to program comprehension) is an economic issue for the industry, making it a formidable teaching/training issue for those that encounter the novice practitioner. The problems of program understanding to the maintenance programmer are magnified by unstructured or poorly structured legacy code. These problems are being tackled by researchers in reverse engineering (Müller94, Storey96) who are attempting to construct tools to help in the comprehension activity.

Program comprehension is an important activity for other industrial practices too. Inspections and code reviews require the individual to deal with an unfamiliar code-base. The reader attempts to understand the product sufficiently to identify defects. Though usually considered an effective mechanism for defect removal, Rifkin and Deimel (Rifkin94) hypothesized that these techniques were sometimes unsuccessful due to the inability of the reviewers to deal with the work product. Interestingly, organizations understand the importance of thorough training in the aspects of inspections, but do not provide any guidance looking for defects or program understanding. What they were able to demonstrate was that reading and understanding is "teachable/learnable" and that training programs to help individuals learn these skills is beneficial in the inspection/review task.

Aside from the importance of comprehension for industrial practice, improvement in code reading should support learning in computer science and software engineering classes. Students are typically given coded examples illustrative of a particular point. The effectiveness of the example is dependent on the ease with which the appropriate information can be extracted by the learner. A learner without strategies may not gain what was expected from the example or may do so only through the expenditure of undue effort. By improving the ability of students to interact with code-based material, instructional practices in these courses may be improved.

It would seem, from the prior narrative, that attention should be paid to issues surrounding code reading and program comprehension leading to instructional strategies and materials. In the next section we will review the research on program comprehension looking for insights into what has been learned regarding the task, and how to apply what is known to the design of activities to aid students.

Theoretical Background

Deimel and Naveda (Deimel90) suggest that practice in program reading is fundamental to student's developing comprehension skills. It does seem intuitive. Yet, the notion of unsupported drill and practice violates an essential operating principle. We want students to develop comprehension skills that improve their effectiveness in various learning arenas. The goal here is to understand enough of the cognitive models of comprehension to plan instructional activities that may improve the students ability to comprehend code well enough to perform various tasks (e.g., test, review, adapt).

Program comprehension is an activity in which the program reader extracts meaning by understanding how a particular program or code fragment performs its task, or what task a particular item performs. A cognitive model of comprehension should help identify viable strategies, the nature of knowledge needed and how such knowledge is used, the representation of knowledge and what types of stimuli evoke retrieval, and how the exploration process is regulated. Several comprehension models have been proposed over the course of the past twenty years. von Mayrhauser and Vans offer a detailed survey of these models (vonMayrhauser95).

All models deal with how the comprehender uses existing knowledge to establish an understanding of the program under consideration. The existing knowledge that is important is technical knowledge and domain knowledge. The technical knowledge, referred to as syntactic knowledge in Shneiderman79 and implementation plans in Soloway88a, is language-dependent and results in what Pennington87a refers to as the program model. The program model deals explicitly with mapping the features of the program to known features of the language and the domain of programming.

Domain knowledge, semantic knowledge in Shneiderman79, deal primarily with the problem domain. One of the comprehension goals is to understand the problem domain items that are used in the program. Apparently comprehension is partially a matter of constructing associations, mappings, between items in the two knowledge areas and the program. This appears to occur at differing levels of abstraction as the comprehension process proceeds.

Supporting Comprehension

With all the study of program comprehension, there is relatively little from the empirical literature towards an educational focus for teaching program comprehension. As mentioned previously, the notion is to have the students practice comprehension by asking them to read foreign code. Those interested in software maintenance have developed approaches that assist, hopefully, the individual in gaining control over large code-bases. A careful review of these strategies or techniques may reveal support for educational comprehension activities. This could indeed be important following the review of theoretical findings. With a model for the understanding process, it is more likely that we can assess the potential of particular techniques. Additionally, theoretical underpinning could also provide the needed structure for scaling the tasks correctly. Clearly the task before us involves finding ways to support the learning of techniques and strategies for program comprehension. It will not suffice to build software tools intended to help the person, rather program comprehension remains, and will remain, a labor-intensive activity that requires improvement at the individual level, not simply automation.

The methods reviewed come from the research perspective of software maintenance. One aspect that deserves review within these areas is what theoretical framework the method claims to support, and how this is explicitly or implicitly indicated.

Reading by stepwise abstraction

This technique was developed by Mills (Linger79) for identifying defects in code documents. During code reading, the reader looks at critical subroutines in the program and determines their function. Once the function is determined then the function, as a behavior, can be used to describe that block of code (abstraction). The reader works through the program hierarchy in this manner assembling abstractions to describe higher level components until the function of the program is determined. This is a bottom-up strategy requiring the understanding of code, and requiring the reader to map the code to suggested problem domain activity.

Basili & Selby (1987) investigated the effectiveness and efficiency of this technique in a professional environment. Their results show that the technique detects more software faults, and has a higher fault detection rate than functional or structural testing.

Active Design Reviews

Parnas and Weiss (1985) suggested a modification of the Fagan inspection process. The new method, called Active Design Reviews, included much more than a reading technique. Active Design Review as described in the article dealt with organizational matters, and conduct of the review. From a reading perspective, reviewers were given a checklist which attempted to focus their attention on particular issues within the document being reviewed. Different reviewers were given different checklists, therefore each reviewer would concentrate on different aspects of the document. Hence, when the review team assembled members of the group brought differing perspectives which were then integrated during the course of the review.

Defect-based Reading

Defect-based reading (Porter95) was developed as a strategy for identifying defects in requirements documents. Defects were categorized. A set of questions was developed for each defect class that would help characterize the class. The questions guides the reader by providing a set of steps (called a scenario) that should be performed during reading. The reader tries to answer the questions presented by the scenario while reading.

Two experiments conducted at the University of Maryland compared defect-based reading to ad hoc reading and checklist-oriented reading. The conclusions were:

1. The defect detection rate of individuals and teams using defect based reading is superior to that obtained with ad-hoc or checklist methods.
2. Scenarios really support reviewers focus on specific defect classes.
3. The checklist method (the current best practice in industry) was no more effective than the ad hoc detection method.
4. On the average, collection meetings contributed nothing to defect detection effectiveness.

Perspective-based Reading

Perspective-based reading (Laitenberger95) is similar to defect-based reading in that different readers are given different tasks. The difference seems to be in the definition used in providing the reader with a perspective. In perspective-based reading readers have different roles-tester, designer and user-that guide the activity. These roles have associated with them operational descriptions (called scenarios). A scenario consists of a set of questions, much like that found in defect-based reading, and activities that guides the reading of the document (sample scenarios can be found in Laitenberger95) ). Perspective-based reading has been applied to the inspection of requirements documents.

"PBR teams were seen to have an improved (statistically significant at the 0.05-level) coverage of defects, for both the generic and NASA documents. It was also shown that individuals using PBR performed better on the generic documents than non-PBR reviewers. There was, however, no significant effect on individual detection rates due to reading technique for the generic documents." (Basili, et al). Basili96 provides a complete discussion of the work in testing the effectiveness of perspective-based reading.

Program Slicing

The slice of a program with respect to a set of program elements S is a projection of the program that includes only program elements that might affect (either directly or transitively) the values of the variables used at members of S. Slicing allows one to find semantically meaningful decompositions of programs, where the decompositions consist of elements that are not textually contiguous.

Program slicing is a technique for visualizing dependencies and restricting attention to just the components of a program relevant to evaluation of certain expressions. Backward slicing reveals which other parts of the program the expressions' meaning depends on, while forward slicing determines which parts of the program depend on their meaning. Slicing helps programmers understand program structure, which aids program understanding, maintenance, testing, and debugging; slicing can also assist parallelization, integration and comparison of program versions, and other tasks.

Ott and Bieman are research ways of connecting slicing and measures of cohesion.

See Wisconsin Program Slicing Project and the Spyder Project at Purdue

Dependency Graphs

Debugging

CACM April 1997 issue has a number of articles devoted to debugging. It is interesting to note from reading the introduction to the issues that the conclusions is, as usual with such reports, that we need more tools to aid the activity. Yet there is a paucity of work related to what those tools should look like and relatively little empirical evidence to help researchers determine the characteristics of those tools.

Viravan94 details the design of a debugging critic. As a critic the system offers advice and guidance as the individual works at removing defects from software. In the process of elaborating the requirements for the critic, the author ran several empirical studies involving a "live" critic. The structure of these experiments, with the methods and material used, may provide an initial direction for constructing comprehension activities where the goal is debugging. It may be important to explore how hypotheses, as discussed in vonMayerhauser, appear in the debugging literature. How can one support both the formation of hypotheses and exploration of hypotheses for confirmation or refutation?

Reading List

Additional References

Basili, V. R. and R. W. Selby. (1987)Comparing the effectiveness of software testing techniques. IEEE Transactions on Software Engineering, 13(12). p.1278-1296.

Parnas, D. L. and D.M. Weiss. (1985) Active design reviews: Principles and practices. 8th International Conference on Software Engineering, pages 215-222.

Comments should be sent to

This document
Created: April 20, 1997
by RLU
Modified: May 29, 1997