In Dìaz-Herrera, J. L. (ed.) Software Engineering Education. Proceedings of the 7th SEI CSEE Conference, San Antonio, Texas, January 1994. LNCS 750. Springer-Verlag.

An Adventure in Software Process Improvement

Laurie Honour Werth

Department of Computer Sciences
The University of Texas at Austin
Austin, TX 78712
lwerth@cs.utexas.edu

Abstract

Software process improvement concepts, applied in an effort to increase productivity and quality in industry are also needed in the academic environment. By teaching process improvement we can prepare students for the future and simultaneously improve our own understanding and teaching of the software engineering In this paper, the Software Engineering Institute's Capability Maturity Model and the ISO9000 standard are introduced. Our Software Engineering class is described, including two different team organizations which were tried. Technical and process job descriptions are given, together with our experience teaching team skills. The lessons learned are combined in the conclusion. Applying software process concepts gives students a meta- model to understand and help manage project complexity. Having explicit technical and process roles improves students learning and , while difficult to quantify, seems to effect the quality of the end- products in a substantive way. Teaching and applying quality improvement ideas is a vital part of preparing students for modern software technology.


Table of Contents

Introduction

Earlier efforts by the author to incorporate software engineering techniques into software engineering courses at the University of Texas at Austin have been previously described [Werth88-92]. Over the years, using higher level software such as MacApp, HyperCard and Oracle in a Macintosh II laboratory, we developed software engineering tools to provide support for various aspects for our class projects such as testing, costing, version control, analysis and design, software process assessment, and defect tracking. A successful collaboration with a local company provided valuable experience for students using industry-strength CASE tools. In 1992, we explored the area of software process improvement, both by teaching the foundations and by applying it directly to the class project.

Our reasoning was, if attention to software process improves the commercial software development environment, then the application of software process techniques should also strengthen the classroom environment. Two positive effects could be expected. First, successful experience with the techniques on the class project would result in students even better prepared to meet the challenges of modern software technology. Second, the use of quality improvement techniques, applied to the software engineering project as currently taught, would be a step in the direction of improving academic education as suggested, for example, by Peter Denning in "Educating the New Engineer" [Denning92]. Software Process

While great strides have been made in developing software engineering methodologies and techniques, companies have been unable to consistently produce high quality software. Stories of software problems appear on a regular basis, for example [Neumann92]. Large amounts of money have been spent on projects that have produced little usable software, as illustrated graphically in the results of a General Accounting Office (GAO) survey shown in Figure 1.

Results of GAO Survey

Figure 1. Results of GAO survey of software contracts

Humphrey [89] has observed that successful projects have been largely based on individual or dedicated team effort rather than on software development methods. We have come to understand that benefits of better methods and tools cannot be realized in undisciplined projects. In the typical ³firefighting² mode in which immature software development organizations function, software quality is compromised to meet unrealistic schedules. Process improvement ideas for software are similar to current business practices based on Total Quality Management (TQM). These ideas were popularized, beginning ten years ago, by books such as Quality is Free. [Crosby79] and In Search of Excellence [Peters82]. Many of the process management and quality improvement concepts have been adapted from the work statistical process control done by W. Edwards Deming and Joseph Juran [Deming86, Juran89].

Based on work done at IBM, Watts Humphrey introduces early software process concepts in his book Managing the Software Process [Humphrey89]. Humphrey¹s work became the foundation of the Software Engineering Institute¹s (SEI) software process improvement program. The SEI adapted this model, providing five maturity levels called the Capability Maturity Model (CMM), shown in Figure 2, which define an effective, staged, progression toward a statistically controlled software process. In more detail, the five maturity levels of SEI's CMM are:

  1. Initial The software process is characterized as ad hoc, sometimes even chaotic. Until under statistical control, no orderly progress is possible.

  2. Repeatable Basic project management processes are established to track cost, scheudule, and functionality. Earlier successes on similar applications can be repeated.

  3. Defined The software process for both managment and engineering activities is documented, standardized and integrated into a standard process for the organization.

  4. Managed Detailed measures of process and product quality are collected, understood and controlled.

  5. Optimizing Continuous process impovement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

As an organization increases in maturity, the difference between targeted and actual results decreases across the project. Development time and cost decrease, while productivity and quality increase. With an objective basis for measuring quality and setting improvement priorities, time and costs become more predictable as rework and errors are removed from the system [Humphrey89]. Paulk [93], describes in detail the series of five stages or levels through which a company must move in order to improve process and resulting product quality. Each level of the model, shown in Figure 2, is the foundation for the next, so it is important not to try to move too quickly or to skip levels.

Capability Maturity Framework

Figure 2. Five levels of software process maturity [Weber91]

Cost savings resulting from the use of the CMM, illustrated in Figure 3, have been substantiatial, resulting in cost savings in the range of 5 to 10 times the cost of making the process improvements. However, companies report that improvements in work environment and motivation have turned out to be an even greater benefit than the cost saving [Dion92, Henry92, Humphreys91, Mays90]. In a mature organization, everyone knows the processes and their own responsibilities. Workers become empowered by helping to develop the process descriptions and by the ability to update processes as needed.

Project internal processes become more visible. Managers know current project status and can monitor quality and customer satisfaction. SEI efforts to assess and help companies improve their software process has recently extended from military contractors to include commercial software developers as well.

NASA                           1982    1985
______________________________________________

Early error detection           48      80
(% errors found)
______________________________________________

Reconfiguration time            11        5
(weeks)
______________________________________________

Reconfiguration effort 		  10.5      4.5
(person-years)
______________________________________________

Product error rate              2.0       0.11
(errors per 1000 lines of code)

Figure 3. On-Board Shuttle Software Improvements [Humphrey91]

Additional material on Software Process and the Capability Maturity Model is available in [Werth93] and other publications from SEI such as [Paulk91; Weber91]. Technical reports and other educational software engineering materials are available on-line using ftp: ftp.sei@cmu.edu. Look in the directory /pubs/documents. General readings in Quality Improvement may be found in Appendix A. Another important software developer certification effort has been undertaken by the International Standards Organization (ISO). Known as ISO9000 [ISO87], the standard applies to all manufacturing efforts. Section three, ISO9000-3 [ISO91], is a modern adaptation of the standard for software development. Developed by the European Community (EC), this standard has already been endorsed by 57 countries. All nations that want to sell products in these countries must have passed a rigorous inspection carried out by a ISO9000 certified auditor. Products with the ISO9000 certification are already beginning to appear in the marketplace.

The ISO9000 audit process is similar to the way SEI's Software Process Assessment (SPA) using the Maturity Questionnaire is carried out. Using a methodology reminiscent of accreditation for academic departments, a questionnaire, describing the use of software engineering principles at the installation is completed. After analyzing the results, a trained team visits the site for a more detailed discussions and analysis. Finally, a report is prepared. The major difference between the two assessments is that SEI's CMM includes a suggested ordering of the implementation of key process areas for overall process improvement, while the ISO9000-3 does not.

Software Process in the Classroom

Software process issues become even more important in the classroom, working with inexperienced students. Using the CMM role definitions and job descriptions in classes can ease students¹ transition to working as a cohesive software engineering team in a professional environment. They have a better appreciation of why software development can be so difficult and it gives students a high-level model for reducing this complexity. The transfer of knowledge from university to industry should begin to include process material in software engineering classes. Even if they do not apply the ideas on a class project, students can gain a clearer idea of the complications inherent in developing large software products and how they can be managed.

Course Description

CS373 is a standard undergraduate software engineering senior project course. The class project for the past two semesters has been a metrics tool, specified by the Software Engineering Institute [SEI91], to collect and analyze software defect reports. The class adopted the name Inferno Engineering and our product was Dante's Defect Tracker, or DDT.

We have evolved a two semester sequence, permitting more time to be spent on process topics as well as providing additional project experience to motivate the need. The first course concentrates on implementing and testing the system designed by the advanced students the previous semester. Because the first course is taught more frequently, this class can be used to maintain and extend earlier versions of the sytem.

Class Project

A significant part of software quality improvement is careful monitoring and controlling of defects. The life cycle of a defect, taken from the User's Manual [DDT92] is shown below in Figure 4. DDT provides a systematic way to enter, track and analyze defects. The defect report screen, in Figure 5, shows the progress of a defect as it moves through the five phases in its life cycle:

Origination: A defect has been found and submitted to the central database.

Evaluation: The report is being reviewed for validity. If valid, it is assigned to a team for repair.

Resolution: The defect is being investigated.

Verification: A repair has been made, and it is being tested.

Closure: The repair has been tested and it has been shown to be successful.

Figure 4. Life Cycle of a Defect [DDT92]

The DDT system is made up of two separate software products: a server which maintains the database and oversees tracking of the defects from a single location, and a reporter application which is used to originate defect reports from any of several reporter sites. In addition to the defect report shown in Figure 5, DDT provides logon security and three editors for entering system administrator(s), projects and their associated team members. Each team member can be given privileges to modify information at any or all of the defect phases and everyone is provided with password protection. A Defect Browser window provides statistical data, the ability to view defect information according to selected viewing criteria, or to sort defects by priority, date, age, status, class and team Five trend graphs, in any of three formats, are also available for viewing summary statistics.

The textbook by Rakos [90], motivated the students, but did not provide sufficient detail for them to accomplish their work. Humphrey's [87] text was also used, as it includes detailed information about the software process in general and defect tracking in particular.

Team Organization

Team organization and scheduling are essential parts of the software process for class projects. A new textbook on team management [Scholtes88] has proved extremely helpful in helping the students organize their meetings and reports.

Students request their team roles by submitting a cover letter with their resume. Most get their first choice. Students are often correct in assessing their job skills, but there is likely to be an imbalance in the relative strengths of the teams.

Tomayko¹s [87] model Software Engineering course was the basis for the first, Spring 1992, semester. Class organization is based on disjoint subteams for design, implementation, testing and evaluation (T&E), documentation, configuration management (CM), quality assurance (QA) and project administration (PA) Students worked from Tomayko's examples as well as the IEEE standards [IEEE87] to develop the process plans and used FileMakerPro to develop simple tools to assist in automating the process. For example, the Project Administration team developed a system to log hours worked, while T&E automated the generation of test case documentation. The Design team wrote the Functional and Design Specifications, working with the Implementation team as they developed a HyperCard prototype.

In the second, Fall 1992, semester, students employed the previous semester's plans and tools to complete the design and implementation of Dante¹s Defect Tracker. However, the process teams were integrated within the technical teams, using a matrix management scheme. See brief job descriptions in Tables 1 and 2. Each of the technical teams (design, implementation, testing, documentation) elects one person to each of the following process teams: project administration, system administration, technical lead, configuration management, quality assurance, and documentation specialist.

This integrated organizational scheme seemed to more closely match the roles that evolved during the earlier, Spring 1992 semester¹s effort, as well as incorporating natural liaison and communication of process procedures within the technical teams. Because of lack of experience, undergraduates seem to find it easier to monitor and impose control when they are a member of the process teams, rather than acting as an unknowledgeable outsider "interfering" with the technical team's efforts.

Technical Team Activities Technical teams met and developed documents appropriate to their role: Functional and Design Specifications by the Design team; software with Programmer¹s Manual by the Implementation team; Test Plans, Cases, Results and Reports by Test and Evaluation team; and an interactive, HyperCard User¹s Manual by the Documentation team. For more details, see Table 1.

Technical Team Job Descriptions Table 1.

Design - primary responsibility is developing aspects of the design as specified by the Systems or User Requirements document. During the pre-design stage, this team could assist in a literature search to explore similar product or problem areas. Primary products are the Requirements or Functional Specification and the Design Specification.

Implementor/Prototyper - primary responsibility is to implement the individual modules of the design and serve as the technical specialist for a particular language and operating system. During the requirements specification and design stages, the implementors could experiment with new software and prototyping tools expected to be needed in the product and to develop tools. They are also responsible for developing and maintaining programmer or system documentation.

Testing - responsible for testing and evaluating prototype, individual modules and subsystems, and for preparing the appropriate test plans. Test results are reported and the item returned to the proper person for changes. - responsible for the creation and execution of test plans to verify and validate the software as it develops.

Documentation - responsible for the creation of user manuals and documentation standards. Other documents might include tutorials/training materials, reference manuals, etc.

The Fall '92 Design team's understanding of the problem had been greatly enhanced by the earlier Spring '92 team's efforts on the Functional and Design Specifications. They were able to work very efficiently on the detail needed to completely specify the user interface, once they comprehended the original high-level design. While the Design team worked, the Implementation team learned the Oracle database and HyperCard interface software. While everyone had been assigned a simple Oracle exercise early in the semester, the product is sufficiently complicated that more time was needed to comprehend it. The second semester team had a relatively easy transition since they had working code from which to learn.

The coding process is so all consuming that it is difficult for students to stand back and take the tester's point of view about their own code. We have had more success with testing in classes where the testing project was different from the earlier analysis and design projects or where the project was to modify an earlier pfoject effort. Where the code to be tested already exists, the students are able to concentrate on their test cases and on managing their results and summary reports, without getting bogged down in the details of the design and/or coding.

Test teams have trouble getting started early enough. Somehow, it is hard to convince students that test cases can be written before working code is developed. Our students have such heavy schedules that it is easy for them to procrastinate, despite project milestones. Also, there do not appear to be good testing tools available for the Macintosh. Improved methods and materials for teaching testing clearly need additional study. The original, interactive User's Manual was not as polished as we have come to expect, though one student developed an innovative, animated tutorial to explain the phases of defect tracking using DDT. The second semester, the Documentation team was extremely professional and produced an impressive User's Manual, complete with artwork by two students in the class.

Developing and enforcing internal documentation standards requires concentrated effort. Using software such as PageMaker helps with the standardization, but at the usual cost of a large learning curve. Even when a knowledgeable student sets up PageMaker templates, it still takes time and training to get other students to use them consistently. A powerful word processor such as Word, can provide sufficient support for high quality documentation, if standards are strongly enforced.

Process Team Activities

During the first semester, the process teams spent most of their time developing planning documents based on the IEEE standards [IEEE87] and Tomayko's [87] students' models. The Project Administrator(s) designed schedules and contract costs using software tools such as MacProject. The Configuration Management team developed the CM Plan, the Quality Assurance team created a QA Plan, while the Documentation Specialists evolved documentation standards. An excellent student developed and implemented a comprehensive System Administrator's Plan for the MacII Software Engineering lab. System Administration continues to be difficult problem both because student needs and Mac skills vary widely and because the equipment is aging rapidly. The second semester process teams enhanced the existing plans to better describe their process function. We added a subteam Technical Lead and (optional) CoPilot who wrote the Project Management Plan and formed the basis of the Configuration Control Board (CCB) membership. Like the original teams, all wrote project legacies to pass along their increased understanding of their process role(s) for future teams.

Process Team Job Descriptions Table 2.

Technical Lead/CoPilot - responsible for the creation of the software document or product. Primary responsibilities include coordinating the document/code development, advising and supervising product development and maintenance.

Project Administrator - responsible for resource allocation and tracking. Primary responsibilities are cost analysis and control, computer and human resource acquisition and supervision. Collects data and issues weekly cost/person reports and the final report.

System Administration - responsible for maintaining the hardware/software configuration for the team or class. Includes monitoring hardware problems, assisting team members with software problems, selecting utilities and other support material.

Configuration Management - responsible for change control. Responsibilities include writing the configuration management plan, calling/conducting change control board meetings, archiving, and preparing product releases.

Quality Assurance Management - responsible for the overall quality of the released product. The customer's representative on the project, oversees all phases. Primary responsibilities include preparing the quality assurance plan, calling and conducting reviews and code inspections, evaluating documents and test plans/results. Acts as a member of the independent group. Primary users of a defect tracking tool such as DDT.

Documentation Specialist - responsible for the appearance, consistency and clarity (not necessarily entering) of the technical documentation for the team of which they are a member.

The second semester, one of the biggest problems seemed to be keeping students from re-inventing the wheel and starting over from the beginning. Some took the attitude that the inherited process plan was impossibly complex. They couldn't possible need all that. Students changed their mind during the semester as they came to appreciate the need for a complete plan. Process plans will continued to be passed along from semester to semester and modified as needed.

Students with work experience in the larger companies that require these processes, understand the need for the plans better than the students who haven't worked yet. Similarly, experienced students can provide models of weekly progress reports and other project activities for the less experienced students. The process organization actually varies somewhat each semester as a function of the experience, skills and motivation of the students in the class. The configuration control board (CCB) consisted of each team¹s configuration management representative, together with the head of quality assurance, the external auditor (the teaching assistant) and CEO (the instructor). Overall project management and coordination gravitated to the CCB meetings, held in the lab before class. Since other teams¹ members were often working in the lab during this time and all teams were represented on the CCB, many questions and issues were resolved quickly and easily. Results were announced or further discussion took place immediately after the CCB meeting during class. The turn-around time for change requests must be kept short and this arrangement seems effective at expediting defect tracking, a bottleneck on an academic schedule.

The second semester we assigned the Technical Lead as the CCB representative, figuring that the Configuration Management person was already busy maintaining backups and controlling versions. This helped the technical lead keep an overall view of team progress. The head of the QA process team is automatically a member of the CCB, as is anyone who's advice is need on a regular basis. Change requests are liberally used to resolve ambiguities and to establish needed standards, so the CCB meetings become the central clearinghouse for settlinging conflicts. All meetings and walkthroughs are open to any student that wants to attend. QA sets up walkthroughs and inspections and is in charge of keeping notes on the meetings. The QA representatives, like the CM and other process teams, form their own group which maintains standards appropriate to their process responsibility. In particular, the Documentation Specialists develop general documentation standards and monitor documents for compliance.

Team and Meeting Skills

The Team Handbook: How to Use Teams to Improve Quality [Scholtes88], provided the single most important increase in team skills. The text provides information on quality improvement generally, but the two chapters on team skills are the primary ones used. Chapter Four provides guidelines for productive meetings, keeping meeting records and forms to be used for agendas, meeting reports and meeting evaluations. Chapter Six discusses team dynamics: forming, storming, norming and performing, together with a discussion of necessary team skills and an extensive discussion of handling team problems. Knowing that everyone is nervous about their first team project and that all teams go through a certain amount of thrashing at the beginning, helps students feel more confident and eases transition to smoothly functioning teams. The ability to run effective meetings, with an agenda, minutes and action items, is a very valuable skill students learn in the course. More motivation to get students started early is needed. The team skills, project roles, standards, and documents must be well- understood early in the semester. Additional time must be devoted to these aspects, despite the tendency to emphasize the technical aspects at the beginning of the course. The process plans, textbooks and student legacies help, but unless required, process will be overlooked in the rush to work on the technical parts with which students are more comfortable.

Team Schedules

We coordinated team efforts though status meetings held each Friday and via walkthroughs of the emerging design which were attended by appropriate representatives of the various teams. The project really began to jell when the Configuration Management Board (CCB) began meeting to discuss change requests. Making the development process visible had a strong, positive effect on the class project. The level of effort rose rapidly, as students began to understand the process and became actively involved in improving the quality of the emerging software product.

At the status meetings, held during class each Friday, one technical team presented their work, while other teams made announcements and reported progress. Everyone in the class acted as either the status meeting moderator or recorder at some point during the course. Agendas and minutes were sent by e-mail to class members. Status meetings worked very well, greatly improving communication and student process learning, as well as reducing the instructor workload. Our industry ³users,² Herb Krasner and Jim Terrel, attended these meetings as their schedules permitted.

Walkthroughs or reviews were held during the week, often after class on Monday or Wednesday, in preparation for the Friday presentation. Attendees include the presenting team and appropriate representatives from related technical and process teams.

Conclusions

Student learning included the usual lessons such as the discovery of the complexity of software development, the need for communication and teamwork, and the importance of configuration management. However, understanding of these issues was considerably deeper than in past semesters with the new course organization.

Even in a class with little work experience, significant learning took place due to the improved process structure. Scholtes' Team Handbook [89] provides excellent information, but additional class time needs to be used to practice team techniques such as walkthroughs. Meeting notes, progress reports and other process documentation must be required. The students themselves suggested assignments, quizzes or other means to insure that everyone learn team skills, roles and process descriptions early in the semester. Further work on improving written job descriptions and process plans should help as well.

Software process techniques lead naturally to an analysis of the educational environment itself. When processes work smoothly, the underlying environment and infrastructure weaknesses become more apparent. Process improvement applied to the students¹ working environment identifies bottlenecks. In a time of limited resources, this is especially helpful for guiding instructors in directing their course organization efforts.

One of the greatest benefits arose from the empowerment of the students, an integral part of the quality improvement process. Students take more responsibility, while at the same time, the instructor is able to dedicate more energy to improving the course as a whole. As the process experts promised, the development process became more visible. We didn't always like what we saw, but we had a much clearer understanding of the things that were and were not working on the project.

It is important that the instructor genuinely gives control to the students and does not simply hide the rule book and expect students to guess the instructor's intended agenda. With grade- conscious students such as ours, empowerment may require re- enforcement for some, but most students relish the opportunity to act in a more professional and creative way than the usual classroom organization permits. Students are have been enthusiastic about the course in the past, but some have complained about the large amount of work involved. Now, however, students work smarter and complain less.

Peer pressure is far more effective than instructor exhortations. Team meeting reports are professional and there are fewer complaints about team members who don't contribute equally to the team effort. It is especially heartening to see a shy student taking the initiative on some part of the project or to watch an "average" student confidently leading the class status meeting. The level of professionalism has risen, as have the student course evaluations.

Class legacies show remarkable wisdom and insight into team and individual personality problems, not to mention considerable creativity and wit. Reading each other's legacies has helped some students to see where they may have contributed to the very problems they had been complaining about. Students' level of awareness of the power of teamwork and commitment to quality makes teaching the class especially rewarding. As usual, the instructor learned more than the students during the semester. Technical analysis, design, and testing techniques are important, but the empowerment, shared learning and more stable environment provided by quality improvement efforts seem to increase learning in the software engineering project course. As the students frequently observed, the sum of the parts is indeed greater than the whole. Learning and applying software process concepts gives students a meta-model to understand and help to manage the complexities of a large software development project. Teaching and applying software process is a vital part of increasing the timeliness of technology transferred as students move from the classroom to industry.

Acknowledgments

This work would not have been possible with the dedication of our users, Herb Krasner and Jim Terrel; Teaching Assistants, Tommy McQuire and Vicki Almstrum, and of course, the Inferno Engineering teams: Spring 1992 Project Manager: Karen Mustard; Principal Architect: David Snider; Design: Randy Clarke, Peter Kraemer; Configuration Management: Stan Stone; Quality Assurance: Steve Moore, Mathew Colquhoun; Validation and Verification: Scott Boland, James Morris; Documentation: Gray Mack, Teshin Lin, Eric Brown.; Project Administration: Chris Owan, Lee Murray, Shinta Tijo. Fall, 1992 Design: Paul Dedeyan, Jon Sligh, Adriane McFetridge, Roger Thompson, Charlie Wood; Implementation: Christoph Borst, Chris Chenault, Robert Dennett, Nick Lauland, Minh Tran, Chris Traynor, Joe Warrington, Jr.; Testing and Evaluation: Scott Boland, Gray Mack, Paul Lewis, John Scalo, Javier Seen; Documentation: W. Perry Copus, Jr., Belinda Easley, Alan Soong.

References

Crosby79 Crosby, P. B. Quality Is Free. New York: McGraw-Hill, 1979.

DDT92 Dante's Defect Tracker Reference Manual. Inferno Engineering, The University of Texas at Austin, CS378 Software Engineering Project, Fall 1992.

Deming86 Deming, W. E. Out of the Crisis. Cambridge, Mass.: MIT Center for Advanced Engineering Study, 1986.

Denning92 Denning, P. J. Educating a New Engineer. Comm. ACM 35, 12 (Dec. 1992), 83-97.

Dion92 Dion, R. Elements of a Process-Improvement Program. IEEE Software (July 1992), 83-85.

Florac92 Florac, W. Software Quality Measurement: A Framework for Counting Problems and Defects. Tech. Rep. CMU/SEI-92-TR-22, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., Sept. 1992.

Henry92 Henry, J. and B. Blasewitz. Process Definition: Theory and Reality. IEEE Software (Nov. 1992).

Humphrey89 Humphrey, W. S. Managing the Software Process. Reading, Mass.: Addison Wesley, 1989.

Humphrey91 Humphrey, W., T. Snyder, and R. Willis. Software Process Improvement at Hughes Aircraft. IEEE Software (July 1991).

IEEE87 Software Engineering Standards. IEEE Press, 1987. ISO 9000.87 Quality Management and Quality Assurance StandardsÑ Guidelines for Selection and Use. 1987. Available from Global Engineering Document, Irvine, Calif. and ISO91 Quality managment and quality assurance standards - Part 3; Guidelines for the application of ISO 9001 to the development, supply and maintenance of software. Available from: American Society for Quality Control, Customer Service Department, P.O. Box 3066, Milwalkee, Wisconsin 53201. 1-800-248-1946, Fax Order to: 1-414-272-1734.

Juran89 Juran, J. M. Juran on Leadership for Quality. New York: The Free Press, 1989.

Mays88 Mays, R., E. Jones, G. Holloway, and D. Studinski. Experiences With Defect Prevention. IBM Systems Journal 29, 1 (1988), 4-32.

Paulk93 Paulk, M. C., B. Curtis, M. B. Chrissis, and C. V. Weber. Capability Maturity Model for Software, version 1.1. Tech. Rep. CMU/SEI-93-TR-24, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., Feb. 1993.

Paulk91 Paulk, M. C., B. Curtis, M. B. Chrissis, E. L. Averill, J. Bamberger, T. C. Kasse, M. Konrad, J. R. Perdue, C. V. Weber, and J. V. Withey. Capability Maturity Model for Software. Tech. Rep. CMU/SEI-91-TR-24, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., Aug. 1991.

Pressman92 Pressman, R. Software Engineering: A Practioner's Approach. 3rd. edition. McGraw-Hill, 1992.

Rakos90 Rakos, J. Software Project Managment for Small to Medium Sized Projects. Prentice-Hall, 1990.

Rumbaugh91 Rumbaugh, J, M. Blaha, W. Premerlani, F. Eddy and W. Lorensen. Object-Oriented Modeling and Design. Prentice Hall, 1991.

SEI91 Measuring Software Quality Using a Problem Management System, Draft for Review. Quality Subgroup of the Software Metrics Definition Working Group, August, 1991.

Scholtes88 Scholtes, P. The Team Handbook: How to Use Teams to Improve Quality. Madison, Wis.: Joiner Associates, 1988.

Tomayko87 Tomayko, J. E. Teaching a Project-Intensive Introduction to Software Engineering. Tech. Rep. CMU/SEI-87-TR-20, ADA200603, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., 1987.

Weber91 Weber, C. V., M. C. Paulk, C. J. Wise, and J. V. Withey. Key Practices of the Capability Maturity Model. Tech. Rep. CMU/SEI-91-TR-25, ADA240604, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., Aug. 1991.

Werth88 Werth, L. H. Software Tools at the University: Why, What and How. Software Engineering Education, Ford, G., ed. New York: Springer-Verlag, Apr. 1988, 169- 186.

Werth89 Werth, L. H. Preparing Students for Programming-in- the-Large. Proc. 20th SIGCSE Tech. Symp. Computer Science Education, Barrett, R. A. and M. J. Mansfield, eds. New York: ACM, Feb. 1989, 37-41.

Werth90 Werth, L. H. Object Oriented Programming and Design Class Projects. J. Object Oriented Programming (Nov./Dec. 1990).

Werth91 Werth, L. H. Industrial-Strength CASE Tools for Software Engineering Classes. Software Engineering Education, Tomayko, J. E., ed. New York: Springer- Verlag, Oct. 1991, 245-256.

Werth93 Werth, L. H. An Introduction to Software Process. Ed Material. CMU/SEI*TBA*** Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., Feb. 1993.

Appendix A: General Readings on Quality

Bridges91 Bridges, W. Managing Transitions: Making the Most of Change. Addison-Wesley Publishing Company, Inc. Reading MA, 1991.

Brooks75 Brooks, F. P. The Mythical Man-Month, Addison-Wesley, Publishing Company, Inc. Reading, MA, 1975.

Byham88 Byham, W. C. with J. Cox. Zapp! The Lightning of Empowerment. Ballantine Books, 1988.

Covey89 Covey, S. R. The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change. Simon and Shuster, Inc. 1989.

Crosby89 Crosby, P. B. Let's Talk Quality, McGraw-Hill, 1989.

Crosby79 Crosby, P. B. Quality Is Free. New York: McGraw-Hill, 1979.

DeMarco82 DeMarco, T. Controlling Software Projects, Yourdon Press, 1982.

DeMarco87 DeMarco T. and T. Lister, Peopleware, Dorset House, New York, NY, 1987.

Deming82 Deming, W. E. Out of the Crisis, MIT Center for Advanced Engineering Study, Cambridge, MA, 1982.

Fisher91 Fisher, R. and W. Ury. Getting to Yes: Negotiation Agreeement Without Giving In. 2nd ed. Penguin Books, 1991.

Gitlow89 Gitlow, H.S. and S.J. Gitlow, A. Oppenheim and R. Oppenheim. Tools and Methods for the Improvement of Quality. Irwin, Inc. i989.

Gitlow87 Gitlow, H.S. and S.J. Gitlow. The Deming Guide to Quality and Competitive Position. Prentice-Hall, Inc. 1987.

Goman87 Goman, C. Change-Busting: 50 Ways to sabotage organizational change. KCS Publishing, 1987.

Harrington87 Harrington, H. J. The Improvement Process: How America's Leading Companies Improve Quality, McGraw- Hill, 1987.

Hirsh90 Hirsh, S. and J. Kummerow. Introduction to Type in Organizations: Individual Interpretive Guide, 2nd ed. Consulting Psychologists Press, Inc., 1990.

Juran88 Juran, J. M. and F. M. Gryna (eds.). Juran's Quality Control Handbook. 4th ed. McGraw-Hill. 1988.

Kanter83 Kanter, R. M. The Change Masters: Innovation for Productivity in the American Corporation, Simon and Schuster, 1983.

Kriegel91 Kriegel, R. J. and L. Patler. If it ain't Broke...BREAK IT! and Other Unconventional Wisdom for a Changing Business World. Warner Books, 1991.

Kroger92 Kroeger O. and J. Thuesen. Type Talk at Work: How the 16 Personality Types Determine Your Success on the Job. Delecorte Press, Bantam Doubleday Dell Publishing Group, Inc. 1992.

Metzger87 Metzger, P. W. Managing Programming People, Prentice- Hall, Englewood Cliffs, NJ, 1987.

Peters87 Peters, T. Thriving on Chaos: Handbook for a Management Revolution, Harper & Row, 1987.

Peters82 Peters, T. J. and R. H. Waterman, Jr. In Search of Excellence: Lessons from America's Best-Run Companies. Harper & Row, 1982.

Radice88 Radice R. A. and R.W. Phillips, Software Engineering: An Industrial Approach, Volume 1, Simon & Schuster, Englewood cliffs, NJ, 1988.

Weisbord88 Weisbord, M. Productive Workplaces: Organizing and Managing for Dignity, Meaning, and Community, Jossey- Bass, 1988.

Senge90 Senge, P.M.The Fifth Discipline: The Art & Practice of The Learning Organization. New York: Doubleday, 1990.

Scholtes88 Scholtes, P. R. The TEAM Handbook: How to Use Teams to Improve Quality. Joiner Associates, Inc. 1988. 3800 Regent St. Box 5445 Madison WI 53705-0445. 800-669- TEAM or 608-238-8134.

Walton86 Walton, M. The Deming Management Method, Perigee Books, New York, NY, 1986.

Companies report that improvements in work environment and motivation have turned out to be an even greater benefit than the cost saving that resulted from the use the CMM [Henry92, Mays90]. In a mature organization, everyone knows the processes and their own responsibilities. Workers become empowered by helping to develop the process descriptions and by the ability to update processes as needed. Project internal processes become more visible. Managers know current project status and can monitor quality and customer satisfaction.

An understanding of these software process improvement programs is vital to prepare computer science students for modern software development technology.

Modeling good work, where it is available, is effective whether one is trying to improve the quality of student work logs, essay exam answers, or other work. Distributing copies of good examples reduce complaints about lower scores on written assignments.

______________________________________________

NASA   				     1982    1985
______________________________________________

Early error detection  			 48      80
(% errors found)

Reconfiguration time (weeks)	     11       5

Reconfiguration effort  		 10.5     4.5
(person-years)

Product error rate      		  2.0     0.11
(errors per 1000 lines of code)

_______________________________________________

On-Board Shuttle Software Improvements

Managing the Software Process

The software task is a process that can be controlled, measured, and improved.

Six steps to improvement:

  1. Understand the current status of development process
  2. Develop a vision of the desired process
  3. Establish a list of actions in priority order
  4. Produce an action plan
  5. Commit resources to execute plan
  6. Repeat
Watts Humphrey, Managing the Software Process, Addison-Wesley, 1989.

Key Process Areas by Maturity Level


Level 2:                                                        Repeatable
        Requirements Management
        Software Project Planning
        Software Project Tracking and Oversight
        Software Subcontract Management
        Software Quality Assurance
        Software Configuration Management

Level 3:                                                        Defined
        Organization Process Focus
        Organization Process Definition
        Training Program
        Integrated Software Management
        Software Product Engineering
        Inter-group Coordination
        Peer Reviews

Level 4:                                                        Managed
        Process Measurement and Analysis
        Quality Management
        
Level 5:                                                        Optimizing
        Defect Prevention
        Technology Innovation
        Process Change Management