7# ]r\\lloooopjpjpjpj ptppHpxpjqD qdDq*qoqqqqqpqqqqqqChapter 6. Software Inspections 6.1 Introduction This chapter describes the modeling of software inspections, in order to illustrate the use of system dynamics modeling for process improvement assessment. The data that is being reported on the use and effectiveness of inspections from across the software industry indicate that they have the potential to reduce software development cycle time. The inspection model described in this chapter has the ability to provide answers to the types of questions, concerning process improvements, posed in Chapter 1. Once the software inspection model is integrated with a software development process model, such as the concurrent incremental software development model, it allows experimentation with a number of variables connected to the inspection process. An overview of the software inspection process is provided in Section 6.2. Section 6.3 provides the software inspection model specification. 6.2 Software Inspections This section provides a general overview of software inspections. Section 6.2.1 describes the software inspection process. Section 6.2.2 describes the expected impact that software inspections will have on software development. Section 6.2.3 discusses the observed impact that software inspections have had at a number of software development organizations. 6.2.1 Description of the Software Inspection Process The inspection process is a disciplined method of finding errors soon after they have been injected into software. Just about anything that can be written down can be inspected. Fagan [Fag76] suggests a number of places in the software life cycle where inspections can be used, including: function level, design, code, test plan, test case, interconnections, fixes/changes, publications, and post-testing inspections. The following paragraphs describe the procedure for inspecting a design, although the aforementioned inspections would proceed in a similar manner. The inspection process starts with the selection of a team of inspectors. The ideal team size consists of three to five members. The team is composed of a moderator, a designer, an implementor, and a tester. Each team member has a specific task and point of view during the inspection process. The moderator is the leader of the team. The moderator should have strong leadership and interpersonal skills. The success or failure of an inspection lies heavily on the competence and preparation of the moderator. The moderator should be selected from a similar, but separate, project. Being independent from the project gives the moderator a greater degree of objectivity than someone closely involved with the project. It is the responsibility of the moderator to set the pace of the inspection and perform supervisory tasks and paperwork. The designer is responsible for producing the design to be inspected. The designer has intimate knowledge of the material to be inspected. The implementor is the programmer who will transform the design into code. After the inspection, the implementor will be able to take the design and begin coding. The tester will write and/or execute test cases that test the design. The tester will be concerned with the testability of the software. Once the team has been selected, the inspection process can begin. There are five steps to a successful inspection: overview, preparation, inspection, rework, and follow-up. The overview step is a presentation by the designer to the rest of the inspection team, giving a broad description of the overall design. The broad description is followed by a more detailed discussion of his particular section of the design to be inspected. His discussion should cover the logic, execution paths, and dependencies of his design. The purpose of the overview step is to familiarize the inspection team with the design to be inspected. At the overview meeting, documentation of the design is distributed to each member of the inspection team. The preparation step is performed separately by each individual. It is a time for each team member to study the design more closely and to understand it in detail. The preparation step is vital to the success of the inspection, as an unprepared inspection team will not be effective in finding flaws. Each inspection team member is educated in the types of errors to look for and where they are most likely to occur. The inspection team is instructed to look at areas where high error rates have occurred in recent inspections. The inspection step requires a meeting of the entire inspection team. The inspection meeting should not exceed two hours in length (due to a decrease in effectiveness after this period of time), but there may be two meetings per day. The purpose of the inspection step is to locate errors in the design; solutions to errors are not pursued. The general format of the inspection consists of a reader (the implementor), reading through the design and describing how it will be implemented. Every piece of logic, as well as every branch, must be covered at least once. The proper reading pace is vital to the success of the inspection. The most effective reading rate is in the range of 90-125 non-commentary source statements per hour [Fag86]. The objective of the inspection step is to find errors. Once an error is found, it is noted by the moderator and its type and severity are classified. Within one day following the inspection, the moderator is responsible for writing a report about the inspection. The report should be based on the forms provided by Fagan [Fag76], and thus, provide statistical information that may be used to aid future inspections and testing. The next step in the inspection process is rework. The rework step is a period of time set aside for the designer to fix the problems discovered during the inspection meeting. The final step is follow-up. Every error or concern voiced during the inspection meeting must be resolved at this time. If these errors are not resolved, they can cost 10-100 times more, in programmer time alone, to fix later in the life cycle [Fag76]. If more than five percent of the design was reworked, the inspection team should reconvene and perform a 100 percent inspection of the design, again. This reinspection is necessary, because it has been shown that one of six error fixes result in new errors [Fag76]. If there was less than five percent rework, the decision to reconvene is left to the discretion of the moderator. If the entire team does not reconvene, the moderator is responsible for inspecting the design, himself. The success of the inspection process depends on strict discipline in following all of the aforementioned steps and having explicit exit criteria stated for each step. Short-cuts in performing the steps and vagueness in enumerating the exit criteria will result in a higher cost of development, due to less effective inspections. It is helpful to have inspection specifications and training classes to educate inspection team members in the proper focus and conduct of inspection teams. 6.2.2 Expected Impact of Software Inspections on Development The inspection process has been around since the early 1970's and, yet, is not in wide use today. There seems to be some resistance on the part of developers and managers to the introduction of the inspection process in the workplace. This section addresses and refutes their fears and concerns by providing a list of expected impacts that software inspections will have on software development. Developers unfamiliar with the inspection process ask questions like: Will the "overhead" of inspections be justifiable? Will management support the use of inspections? Will the data collected be worth the effort? Will the data collected be used as part of an individual's performance review? [Fow86]. Managers have those same questions and more: How will inspections be used when specification documents are informal and incomplete, if nonexistent, and schedules are tight and seemingly inflexible? [Fow86]. The following list of observations should serve to educate potential users of inspections and alleviate their concerns voiced above. Inspections find 60-90 percent of all defects uncovered during the software life cycle [Fag86]. Inspections should be thought of as part of the development process, and time must be set aside accordingly. Once this is done, inspections can have a significant improvement in the development organization's ability to meet internal schedules [GHP86]. Inspections put clout behind completed milestones, and add visibility which aids in management's tracking. Proper use of inspections can even shorten life cycle [Fag76]. Participants in the inspection team get a high degree of product knowledge, which leads to higher productivity [Fag76]. Rework can lead to lower productivity, but if done early, there is less rework and this leads to an increase in productivity [Fag76]. Rework done before coding and before unit test is 10-100 time less expensive than if done after this point [Fag76]. Due to real-time turn-around in error discovery, the programmers find out what types of errors they are most likely to make, their quantity, and how to find them. This error-awareness leads to an improvement on the same project, with developers injecting fewer errors as they realize the mistakes they have been making [Fag76]. Slower programmers show a great deal of improvement when using inspections [BL89]. Inspection results are not used in an individual's performance review, because it tends to lead to less effective inspections [Fag76]. Design estimates improve, because designers are asked to estimate the number of lines of code during design, and asked to count the number of actual lines during inspection. In time, estimates should become more accurate [Fag76]. In smaller companies or groups, where human resources are limited, it has been demonstrated that team sizes as small as two (although three to five is preferred) can still be beneficial [BL89]. Strict adherence to the inspection process not only provides data for relation of inspection results to end-product quality, but also yields significant reductions in first-time development costs, long-term maintenance costs and product errors found after development [GHP86]. Inspection leads to a reduction in maintenance, which allows for the redirection of programmers to work off the application backlog, which is reputed to contain at least two years of work at most locations [Fag86]. The one concern not directly or indirectly addressed by the findings of researchers, as listed above, is the question of how inspections will be used when specification documents are informal and incomplete, if nonexistent. In the case of exploratory development, this situation, informal and incomplete specifications, may arise. In that case, after new discoveries have been made and documented, inspections should be held. Inspections can be effective on any written document [Fag76]. 6.2.3 Observed Impact of Software Inspections on Development The statements above describe a number of benefits gained by using the inspection process. What follows is a list of figures obtained by industry upon implementing software inspections. In tests at Aetna and IBM, inspections found 82 and 93 percent, respectively, of all defects (that would cause malfunction) detected over the life cycle of the products [Fag86]. Aetna Life and Casualty obtained a saving in programmer resources of 25 percent by using inspections [Fag76]. Fagan's use of inspections on design and code at IBM found 82 percent of the defects uncovered; 18 percent were found in later tests. Coding productivity increased 23 percent, because less rework was needed to correct defects [Fow86]. Inspections also produced 38 percent fewer errors than walk-throughs [Fag76]. Bell Labs, with more than 200 members of technical staff, increased its productivity by 14 percent from one release to the next by using improved project phasing and tracking mechanisms, including inspections and reviews. Early software fault-density data showed a tenfold improvement in quality [Fow86]. After AT&T used inspections on the 5ESS switch project they concluded that reviews and inspections are cheap and effective mechanisms for early defect detection. Defects detected in inspections cost an average of ten times less to fix than those found during development, but outside inspections and reviews [Fow86]. Eisele and Rathburn used inspections during the testing phase of "generic1" and found that "it is about 20 times more effective to find bugs in the tests during an inspection than to find them during lab sessions" of testing. Inspections of test documents cost 2 percent of the total testing effort [Fow86]. R. Larson reported that for a product of 20 KLOC, inspection resulted in savings of more than 85 percent in programmer time by detecting the major defects by inspection, as opposed to finding them during functional variation testing. He also found that inspection uncovered a need to modify approximately 30 percent of the functional matrices representing test coverage. Inspections also detected 176 major defects in the test plans and test cases. In other words, in 176 cases testing would have missed testing critical functions or tested them incorrectly [Fag86]. Russell used inspections at Bell-Northern Research on a 2.5 million LOC project and found one defect for every man-hour invested. This was reputed to be two to four times faster than detecting code errors by execution testing. Each defect in software released to customers and subsequently reported as a problem required an average of 4.5 man-days to repair. Therefore, each hour spent on inspection avoided an average of 33 hours of subsequent maintenance effort, assuming a 7.5-hour workday. He found inspections to be a major return on investment. It not only canceled inspection's original cost, but also provided a sizable saving in effort over the total project life cycle. Best of all, customers saw higher quality in the delivered product [Rus91]. 6.3 Modeling Software Inspections This section specifies a software inspection model. Section 6.3.1 specifies the purpose of the software inspection model and its intended audience. Section 6.3.2 specifies the model boundary, what is included and excluded from the model. Section 6.3.3 specifies the structure of the model. Appendices D and E contain the design and implementation of the software inspection model, respectively. The model was implemented using ithink, a modeling tool used to create system dynamics models [ith94]. 6.3.1 Model Purpose and Audience The purpose of the software inspection model shall be to allow researchers and software project managers to manipulate a number of variables connected to the inspection process, in order to understand their impact on software development cycle time. Direct manipulation of the following variables shall be allowed: The time spent on each inspection task per unit of work to be inspected (e.g., the preparation rate and the inspection rate); The size of the inspection team; The percent of errors found during inspection; The percent of tasks that undergo reinspection; and The defect prevention attributable to the use of inspections. The software inspection model must be integrated with a model of the software development process before it may be used. Once the two models are integrated to form a functioning unit, a simulation may be run to examine the effect of inspections on software development cycle time. Further manipulation of the software inspection model's input variables between simulations may provide further insight into possible scenarios for reducing software development cycle time. 6.3.2 Model Boundary The boundary of the model defines what the model includes and/or excludes, with respect to its underlying theory and subsequent simulation. Sections 6.2.2 and 6.2.3 discussed the benefits that can be expected when employing software inspections. This section briefly describes what the model will include and exclude in its attempt to represent software inspections. The software inspection model shall be based on the following assumptions: Time allocated to software inspections takes time away from software development; A larger inspection team will consume more man-hours per inspection than a smaller team; Software inspections find a high percentage of errors early in the development life cycle; and The use of inspections can lead to defect prevention, because developers get early feedback with respect to the types of mistakes they are making. The software inspection model shall not incorporate the following: Software developers achieve higher productivity, due to an increase in product knowledge acquired through the software inspection process; Software developers achieve improved design estimates, due to the attention paid to size estimates during inspection; and Software inspections lead to increased visibility of the amount of work completed. The model shall also exclude the interaction that the inspection team size and the time spent performing inspection tasks have on the percent of errors found during inspection. The inspection team size, the time spent on inspection tasks, and the percent of errors found during inspection will be set at the beginning of a model simulation, according to historical metrics, and may be altered during the simulation. 6.3.3 Model Structure Specification In order to judge the impact that software inspections have on software development cycle time, the software inspection model must be integrated into a model of the software development process. Once integrated, the software inspection model will impact a number of elements in the software development process model. Figures 6.1 and 6.2 are an incomplete, but representative, view of the integrated model. Before discussing the details of Figure 6.1 and Figure 6.2, a brief discussion of flow diagrams, a notation used to represent system dynamics models, is in order.  Figure 6.1: Flow Diagram of Simplified Inspection Process Steps.  Figure 6.2: Flow Diagram of Inspection's Impact on Errors. Flow diagrams are composed of levels, material flows, rates, auxiliaries, and information links. Levels, depicted as rectangles, represent the accumulation of a material in a system. For example, work products and errors are materials that reside in levels in Figure 6.1 and Figure 6.2. Material flows, depicted as hollow arrows, indicated the ability for material to flow from one level to another. Material flows connected to clouds indicate a flow to, or from, a portion of the system not pictured. Rates, depicted as circles attached to material flows, represent the rate of material flow into, and out of, a level. For example, "error detection rate" in Figure 6.2 controls the rate at which "Potentially Detectable Errors" are detected. Auxiliaries, depicted as circles, aid in the formulation of rate equations. In Figure 6.2, "defect prevention" is an auxiliary that affects the "error generation rate." Information links, depicted as arrows, indicate the flow of information in a system. Information about levels, rates, and auxiliaries are transferred using information links. Information links, unlike material flows, do not affect the contents of a level. Figure 6.1 represents the process steps and effort involved in inspecting work products. Each rate in Figure 6.1 requires that manpower be consumed, in order to move work products from one step to the next. Manpower shall be allocated to the inspection process steps, such that no bottlenecks will occur in the inspection process. Figure 6.2 shows an incomplete, but representative, view of the interface between the software development process model and the process improvement model, that is shown abstractly in Figure 4.1. Figure 6.2 represents the modeling of errors in the software development process model and illustrates the impact inspections will have on error generation and error detection in that model. The impacts that software inspections shall have on software development are: software inspections consume development man-hours, errors are less expensive to find during software inspection than system testing, software inspections promote defect prevention, and software inspections reduce the amount of defect regeneration. The first impact that software inspections shall have on software development is in the allocation of man-hours for development. Software inspections require that time be set aside for them to be done properly. The time consumed by software inspections is time that cannot be used for other activities, such as writing software. The amount of time that an inspection consumes is based on the size of the inspection team and the time spent on inspection tasks, e.g., the preparation rate and the inspection rate. Figure 6.1 is a simplified view of the steps that must be taken to perform inspections. It implicitly models the effort expended to move work products through all of the process steps associated with inspection; effort that takes time away from development activities. The second impact that software inspections shall have on software development is in the detection of errors. Errors are found early in the life cycle when using inspections. Errors are less expensive to find and correct during development than during testing. This impact is not explicitly shown in Figure 6.2, except that a higher error detection rate will increase the number of errors found early in the life cycle, rather than later during testing. The third impact that software inspections shall have on software development is in the prevention of defects. Successful inspections attack the generation of future defects. Developers that are involved with inspections learn about the types of errors that they have been making, and are likely to make, during the project. This feedback, about the errors that developers are making, leads to fewer errors being made during the project. Figure 6.2 shows defect prevention, due to inspections, impacting the rate of error generation. The fourth impact that software inspections shall have on software development is a reduction in the regeneration of defects, errors resulting from undetected defects. Defects that remain undetected often lead to new errors being generated. For example, undetected design errors lead to coding errors, because developers are coding to a flawed design. Software inspections detect errors early in the life cycle, thus, reducing the amount of defect regeneration. Figure 6.2 indicates that the "Errors Escaping Detection" impact the "error generation rate." The number of "Errors Escaping Detection" is dependent on the "preparation rate" and the "inspection rate," as shown in Figure 6.2. 6.4 Summary The four impacts that software inspections have on software development, specified above, are the foundation for the theory upon which the software inspection model is based. The design and implementation of the theory, using system dynamics techniques, are shown in detail in Appendices D and E, respectively. The integrated system dynamics model can be used to evaluate the impact that software inspections have on software development cycle time. vy{|^i v_ _ _0'rF, Geneva .+R WP Awaiting Inspection0'F)WP at Preparation0'bF)|WP at Inspection0'F)eWP at Follow Up0'*FS)h WP CompletedP:R(aschedule delay"5J"7Jp0<506<7":"5"3P:.RF)tpreparation rate"5J"7Jp0\<b5\0\6b<\7\"::"5:"37P:R)~ pass rate"56"76p0<506<7":"5"3P:R)Yfollow up rate"56"76p0$<*5$0$6*<$7$": "5 "3 0(WP to be ReinspectedP3K+^A fail rate"EmV"EkT"m"kp"?"?"<P(reinspection rework rate""""pEKKKEKK"""`.8$`.8)`3=)<`3=$P:BRZ(a(development rate"5*B"7*Bp0l<r5l0l6r<l7l":N"5N"3KN{j@{j0  0'F, Geneva .+ Potentially Detectable Errors0':Fc)Detected Errors0'F)Reworked Errors0(Errors Escaping DetectionP\t(Uerror escape rate"E@"E@p"h"h"ePE](8error detection rate"@^"B^p;4G:@4;4A:G4B4"E"@">PE]) rework rate"@b^"Bb^p;G@;AGB"E"@">`9C$`9C)`>H)<`>H$PE`]x(8:error generation rate"@*~"B*~p;G@;AGB"El"@l">iPt@X(%defect preventionP\Ztr(Bpreparation rateP*B(inspection rate`k-^2aUlcz2P`SJXaMZ[h2PpHvN`P: aJX=2P_We]`Uv$aes2PiVo\`D}5QaQ _g2P06`qV#ajxn2P(.der" %%Creator:f:lHKHLHH]e @ @a @]@  17l bF!!!"#-#$/$%&E&'({)*k,V,-N-O..s/02%3\588::< < <<<==U쿹ع瑹 ! ! !  ! ! !  !  !  !  ! ! !!!!!!!!! !!!7=U?.?CAAAUABBBBCwCDHEF HJHKHMHNHHHHHHMlQTVcX}[2[>];ͧ;!!! ! ! !{$ !$! !$ ! $!!! !  ! !!!! listNormal continuedquote 1quote 2 reference table titlemetricssublistbulletcodefigure captionabstract paper titleabstract headingauthorsfigure   $$   @    pp0hp D$ 0 h8 h pp  ` L8 L8   h@@ @!@3   \]k'{/9@FJPR[=\d:e2fghi@j]kTl=mnope3=U]45 !" HH(FG(HH(d'@jb=/pRdHD-:LaserWriter 7.2 #Macintosh HD:Chapter 7 - Validation\Times Helvetica(pChapter 6 - Inspectionscycle time reduction John D. Tvedt1.0chapter 6, cycle time reductionjim collofello