Experimental Software Engineering Group

Check the suffix before using. Files are either pdf or postscript format.
Seaman, C. B. & V. R. Basili (199?) Communication and Organization in Software Development: An Empirical Study (PDF)

Abstract: The empirical study described in this paper addresses the issue of communication amongmembers of a software development organization. The independent variables are variousattributes of organizational structure. The dependent variable is the effort spent onsharing information which is required by the software development process in use. Theresearch questions upon which the study is based ask whether or not these attributes oforganizational structure have an effect on the amount of communication effort expended. In addition, there are a number of blocking variables which have been identified. Theseare used to account for factors other than organizational structure which may have aneffect on communication effort. The study uses both quantitative and qualitative methodsfor data collection and analysis. These methods include participant observation, structuredinterviews, and graphical data presentation. The results of this study indicate that severalattributes of organizational structure do affect communication effort, but not in a simple, straightforward way. In particular, the distances between communicators in the reportingstructure of the organization, as well as in the physical layout of offices, affects howquickly they can share needed information, especially during meetings. These resultsprovide a better understanding of how organizational structure helps or hinderscommunication in software development.

Basili, V. R., G. Caldiera, H. D. Rombach (1994) The Experience Factory

Abstract:This article presents an infrastructure, called the experience factory, aimed at capitalization and reuse of life cycle experience and products. The experience factory is a logical and physical organization, and its activities are independent from the ones of the development organization. The activities of the development organization and of the experience factory can be outlined in the following way:

Basili, V. R., G. Caldiera, H. D. Rombach (1994) The Goal Question Metric Approach (PDF)

Abstract:As with any engineering discipline, software development requires a measurement mechanism for feedback and evaluation. Measurement is a mechanism for creating a corporate memory and an aid in answering a variety of questions associated with the enactment of any software process. It helps support project planning (e.g., How much will a new project cost?); it allows us to determine the strengths and weaknesses of the current processes and products (e.g., What is the frequency of certain types of errors?); it provides a rationale for adopting/refining techniques (e.g., What is the impact of the technique XX on the productivity of the projects?); it allows us to evaluate the quality of specific processes and products (e.g., What is the defect density in a specific system after deployment?). Measurement also helps, during the course of a project, to assess its progress, to take corrective action based on this assessment, and to evaluate the impact of such action.

Goal-Driven Definition of Product Metrics Based on Properties

Abstract: Defining product metrics requires a rigorous and disciplined approach, because useful metrics depend, to a very large extent, on one's goals and assumptions about the studied software process. Unlike in more mature scientific fields, it appears difficult to devise a "universal" set of metrics in software engineering, that can be used across application environments. We propose an approach (GQM/MEDEA: GQM/MEtric DEfinition Approach) for the definition of product metrics which is driven by the experimental goals of measurement, expressed via the GQM paradigm, and is based on the mathematical properties of the metrics. This approach integrates several research contributions from the literature into a consistent, practical and rigorous approach. The approach we outline should not be considered as a complete and definitive solution, but as a starting point for discussion about a product metric definition approach widely accepted in the software engineering community. At this point, we intend to provide an intellectual process that we think is necessary to define sound and useful software product metrics. A precise and complete documentation of such an approach will provide the information needed to make the assessment and reuse of a new metric possible. Thus, product metrics are supported by a solid theory which facilitates their review and refinement. Moreover, their definition is made less exploratory and, as a consequence, one is less likely to identify spurious correlations between process and product metrics.

Property-based Software Engineering Measurement

Abstract: Little theory exists in the field of software system measurement. Concepts such as complexity, coupling, cohesion or even size are very often subject to interpretation and appear to have inconsistent definitions in the literature. As a consequence, there is little guidance provided to the analyst attempting to define proper measures for specific problems. Many controversies in the literature are simply misunderstandings and stem from the fact that some people talk about different measurement concepts under the same label (complexity is the most common case).

There is a need to define unambiguously the most important measurement concepts used in the measurement of software products. One way of doing so is to define precisely what mathematical properties characterize these concepts, regardless of the specific software artifacts to which these concepts are applied. Such a mathematical framework could generate a consensus in the software engineering community and provide a means for better communication among researchers, better guidelines for analysts, and better evaluation methods for commercial static analyzers for practitioners.

In this paper, we propose a mathematical framework which is generic, because it is not specific to any particular software artifact, and rigorous, because it is based on precise mathematical concepts. This framework defines several important measurement concepts (size, length, complexity, cohesion, coupling). It does not intend to be complete or fully objective; other frameworks could have been proposed and different choices could have been made. However, we believe that the formalisms and properties we introduce are convenient and intuitive. In addition, we have reviewed the literature on this subject and compared it with our work. This framework contributes constructively to a firmer theoretical ground of software measurement.

On the Application of Measurement Theory in Software Engineering

Abstract: Elements of measurement theory have recently been introduced into the software engineering discipline. It has been suggested that these elements should serve as the basis for developing, reasoning about, and applying measures. For example, it has been suggested that software complexity measures should be additive, that measures fall into a number of distinct types (i.e., levels of measurement: nominal, ordinal, interval, and ratio), that certain statistical techniques are not appropriate for certain types of measures (e.g., parametric statistics for less-than-interval measures), and that certain transformations are not permissible for certain types of measures (e.g., non-linear transformations for interval measures). In this paper we argue that, inspite of the importance of measurement theory, and in the context of software engineering, many of these prescriptions and proscriptions are either premature or, if strictly applied, would represent a substantial hindrance to the progress of empirical research in software engineering. This argument is based partially on studies that have been conducted by behavioral scientists and by statisticians over the last five decades. We also present a pragmatic approach to the application of measurement theory in software engineering. While following our approach may lead to violations of the strict prescriptions and proscriptions of measurement theory, we demonstrate that in practical terms these violations would have diminished consequences, especially when compared to the advantages afforded to the practicing researcher.

Measuring the Impact of Reuse on Quality and Productivity in Object-Oriented Systems

Abstract: This paper presents the results of a study conducted at the University of Maryland in which we assessed the impact of reuse on quality and productivity in OO systems. Reuse is assumed to be a very effective strategy for software industry to build high-quality software. However, there is currently very little empirical information about what we can expect from reuse in terms of productivity and quality gains. This also applies to OO development which is supposed to facilitate reuse. Our experiment is one step towards a better understanding of the benefits of reuse in an OO framework, considering currently available technology. Data was collected, for four months, on the development of eight medium-size management information systems with equivalent requirements. All eight projects were developed using the Waterfall Software Engineering Life Cycle Model, an Object-Oriented (OO) design method and the C++ programming language. This study indicates significant benefits from reuse in terms of reduced defect density and rework as well as increased productivity.

Domain Analysis of the Reuse of Software Development Experiences

ABSTRACT: We need to be able to learn from past experiences so we can improve our software processes and products. The Experience Factory is an organizational structure designed to support and encourage the effective reuse of software experiences [Bas94]. This structure consists of two organizations which separates project development concerns from organizational concerns of experience packaging and learning. The experience factory provides the processes and support for analyzing, packaging and improving the organization's stored experience. The project organization is structured to reuse this stored experience in its development efforts. However, a number of questions arise:
This paper describes approaches to help answer these questions. We propose both quantitative and qualitative approaches for effectively reusing software development experiences.

AINSI - An Inductive Software Process Improvement Method: Concrete Steps and Guidelines (postscript)

Abstract: Top-down approaches to process improvement based on generic "best practice" models (e.g., CMM, TRILLIUM, BOOTSTRAP, SPICE) have became popular. Despite the idiosyncrasies of each of these approaches, they share some common characteristics: all of them are based on numerous assumptions about what are best practices, and about the business goals of organizations and the problems they face. Other organizations, like the Software Engineering Laboratory of the NASA Goddard Space Flight Center, HP and CRIM in Canada, have adopted the Quality Improvement Paradigm (QIP). The QIP stipulates a more bottom-up and inductive approach to process improvement. The focus of this paradigm is to first understand what processes exist in the organization and to determine what causes the most significant problems. Based on this, opportunities for improvement are devised, and empirical studies are conducted to evaluate potential solutions. In this paper, we present a method, named AINSI (An INductive Software process Improvment method), which defines general but concrete steps and guidelines for putting in place the QIP. This method is the result of the collective experiences of the authors and integrates many lessons learned from process improvement efforts in different environments. It also integrates many complementary techniques such as qualitative analysis, methods for data collection (e.g., the Goal/Question/Metric paradigm), and quantitative evaluation.

F. B. Abreu and W. Melo (1996) Evaluating the Impact of Object-Oriented Design on Software Quality.
Accepted for Publication 3rd Int'l S/W Metrics Symposium, March 1996, Berlin, Germany.

Abstract: This paper describes the results of a study where the impact of Object-Oriented design on software quality characteristics is experimentally evaluated. A suite of metrics for OO design called MOOD was adopted to measure the use of OO design mechanisms. Data collected on the development of eight small-sized information management systems based on identical requirements was used to assess the referred impact. The results obtained in this experiment show how OO design mechanisms like inheritance, polymorphism, information hiding and coupling influence quality characteristics such as defect density and rework. Predictive models based on OO design metrics built in this study are also presented.

V. Basili (1993) The Experimental Paradigm in Software Engineering. appears in Experimental Software Engineering Issues: Critical Assessment and Future Directives, Proceedings of Dagstuhl-Workshop, edited by H. Dieter Rombach, Victor R. Basili , and Richard Selby,, September 1992, published by Springer-Verlag, #706, Lecture Notes in Computer Software, August 1993. (pdf) or (ps)