To frame our discussion, consider:
What quality perspectives can be applied to software?
What quality factors are useful in describing characteristics of the software solution?
What is the purpose of software quality assurance?
What are the responsibilities of an SQA team?
Quality can be identified but not defined. " As an affirmative - "I know it when I see it!!", and a negative - "I know when its missing!!".
The user view of quality is in the context of use. This view gauges quality with regard to fitness for use.
Manufacturing process takes precedence over product. Here quality is assessed based on performing the right tasks to construct the product assuming such acts result in better products.
The product view is conformance to specification. We build what we wanted to build as detailed in the specification.
Equates quality with what customers are willing to pay. This view provides the foundation for trade-offs. This view is typical later in product development cycles when change requests are handled.
Type | GENERAL DESCRIPTION |
| System | Compilers, editors, file management utilities, and operating systems are some examples. Characterized by heavy interaction with hardware, multiple users, and concurrent operation of processes |
| Real-Time | Measures/analyzes/controls real-world events as they occur. |
| Business | Restructures existing data in order to facilitate business operations or management decision making. |
| Engineering and Scientific | Generally considered the number crunching software. Currently though, new applications would include CAD and simulation. |
| Embedded | Resides in ROM and is used to control products and systems for the consumer and industrial markets. |
| Personal Computer | This category includes the traditional "individual productivity tools" as well as games and entertainment software. |
| Artificial Intelligence | AI software makes use of nonnumerical algorithms to solve complex problems. These are sometimes referred to as expert systems or knowledge-based systems. |
TYPE OF SOFTWARE | P O R T A B I L I T Y | R E L I A B I L I T Y | E F F I C I E N C Y | P E R F O R M A N C E | R O B U S T N E S S | C O R R E C T N E S S |
| System | N | Y | Y | Y | Y | Y |
| Real-Time | N | Y | Y | Y | Y | Y |
| Business | Y | Y | N | N | N | Y |
| Engineering and Scientific | Y | Y | N | N | N | Y |
| Embedded | N | Y | Y | Y | Y | Y |
| Personal Computer | Y | N | N | N | N | Y |
| Artificial Intelligence | Y | N | N | N | N | Y |
| P | R | E | H | T | U | M | |
| portability | X | X | X | ||||
| reliability | X | X | |||||
| efficiency | X | X | X | X | X | X | |
| human engineering | X | X | X | X | X | ||
| testability | X | X | X | X | X | ||
| understandability | X | X | X | X | |||
| modifiability | X | X | X | X |
Group | Primary Objective | Rank on Primary | Secondary Objective | Rank on Secondary | Core Used | Output Readability | Program Readability | Statements Used | Time (hours) |
| 1 | minimum core | 1 | minimum statements | 2 | 1996 | 5 | 4 | 52 | 74 |
| 2 | minimum execution time | minimum prog. hours | 5 | 6768 | 4 | 5 | 100 | 50 | |
| 3 | output readability | 1 | program readability | 1 | 13438 | 1 | 1 | 166 | 30 |
| 4 | program readability | 2 | output readability | 2 | 4690 | 2 | 2 | 90 | 40 |
| 5 | minimum statements | 1 | minimum core | 2 | 2486 | 6 | 3 | 33 | 30 |
| 6 | minimum prog. hours | 1 | program read ability | 6 | 8548 | 3 | 6 | 126 | 28 |
1. The later in the life cycle that an error is detected the more expensive it is to repair.
2. Errors remain latent and are not detected until well after the stage at which they are made.
3. There are numerous requirements errors.
Estimates indicate that 56% of all errors are errors during the requirements stage.
4. Requirements errors are typically nonclerical. 77% requirements errors on A7E were nonclerical.
The software industry relies heavily on testing, executing the program using prepared data, in an attempt to find defects. In fact, software developers plan and conduct several types of testing during the course of product development. There is unit testing, integration testing, and acceptance testing. Each of these testing strategies employs a variety of techniques for developing test cases.
In 1976, M. Fagan called in to question the robustness of execution-based testing as a defect detection strategy. He introduced the inspection process into software development at IBM. The idea was simple have a group read the artifact (code, requirement, design, test plan, etc.) looks for problems. For the past twenty years data has accumulated regarding the effectiveness of this strategy for defect detection. The data continues to show that informal and formal reviews can be more effective than execution-based testing in finding defects.
This is an informal technique involving the author of the artifact and one or more other developers familiar with the development activity. Meetings are loosely defined, lasting 30-60 minutes. Walkthroughs do not require documentation of the meeting results.
In code reading two or more developers read the code independently, then meet with author individually to discuss it. There is typically no process defined for code reading, nor is there a documentation expected as an outcome.
The code inspection was originated by Fagan. Fagan defined a systematic process for the activity. The process definition included planning, preparation , inspection, rework and followup. Included in the process definition were specific roles for the participants.
During the inspection meeting the author/reader paraphrases the material, while the participants ask question that may lead to the discovery of defects.
The author conducts an informal meeting with readers to establish background and distributed material. The reviewers go through the material individually using a checklist as a guide. The author collects the checklists and consolidates the results. The summary results are presented at a group meeting.
This method is more directed than code reading. Reviewer receive a checklist. The questionnaire focuses the reviewer's attention on a particular set of issues. Different reviewers get different questionnaires, thus, reviewers concentrate on different sets of issues. The author meets with the reviewers individually to review their findings.
The starting point for defect-based reading is a model of possible defects in requirements documents. For each defect class a set of questions was developed that would characterize the defect class, e.g. the class of data type inconsistencies. The questions also characterize a set of steps that should be performed while reading. The set of steps is called a scenario. While reading the document and following the different steps, the reader tries to answer the questions provided in the scenario.
Defect-based reading has been shown to be more effective than ad-hoc reading and checklist-oriented reading. The message from the research appears to be the use of scenarios.These seem to support reviewers' focus on specific defect classes.
Current work in defining processes for reviews includes calling into question the viability of group-based techniques which require meetings. Porter and Johnson (Porter97) found that meetingless methods can be as effective as meeting-based ones. Individual strategies find more issues to resolve but these have a higher false positive (identified problems that later were determined not to be defects) and duplication rates. The duplication rates can be reduced by specializing the roles during the process.
In the software engineering community there is considerable work in progress related to supporting technical reviews. Some the effort relates to supporting clerical functions of distribution and defect report collection and rework tracking. Other work is aimed at synchronous distributed reviews where the review does is not required to be in the room (or the same country).
Software Quality Assurance is a planned and systematic pattern of actions required to ensure quality in software.
Go To Lecture [Outline] [Overview]
Go To [Course Outline]