Papers included in this bibliography are software cost estimation, software economics, models of quality, cost of software quality, and cost models for design and development.
Abstract:
Modularity in design, that is, breaking apart integrated systems into components linked by standard interfaces, has been a central development in many of the recent advances in R&D and manufacturing. In this paper, we analyze the benefits of making a product modular. We show that under particular conditions, modularity leads to (1) higher efficiency of R&:D; (2) a higher rate of technological change; and (3) greater variety in the final product line. We also describe the costs of pursuing a modular design strategy, and discuss conditions that might make modularity unprofitable.
Abstract:
In this paper we reconcile two opposing views regarding the presence of economks or disconomies of scale in new software development. Our general approach hypothesizes a production function model of software development that allows for both increasing and decreasing returns to scale, and argues that local scale economies or diseconomies depend upon the size of projects. Using eight different data sets, including several reported In previous research on the subject, we provide empirical evidence in support of our hypothesis. Through the use of the nonparametric DEA technique we also show how to identify the moot productive scale size that may vary across organizations.
Abstract:
This paper summarizes several classes of software cost estimation models and techniques: parametric models, expertise-based techniques, learning-oriented techniques, dynamics-based models, regression-based models, and composite-Bayesian techniques for integrating expertise-based and regression-based models.
Experience to date indicates that neural-net and dynamics-based techniques are less mature than the other classes of techniques, but that all classes of techniques are challenged by the rapid pace of change in software technology.
The primary conclusion is that no single technique is best for all situations, and that a careful comparison of the results of several approaches is most likely to produce realistic estimates.
Abstract:
The fundamental goal of all good design and engineering is to create maximal value added for any given investment. There are many dimensions in which value can be assessed, from monetary profit to the solution of social problems. The benefits sought are often domain-specific, yet the logic is the same: design is an investment activity. Software economics is the field that seeks to enable significant improvements in software design and engineering through economic reasoning about product, process, program, and portfolio and policy issues. We summarize the state of the art and identify shortfalls in existing knowledge. Past work focuses largely on costs, not on benefits, thus not on value added; nor are current technical software design criteria linked clearly to value creation. We present a roadmap for research emphasizing the need for a strategic investment approach to software engineering. We discuss how software economics can lead to fundamental improvements in software design and engineering, in theory and practice.
Notes: JO
Abstract:
The accurate prediction of software development costs may have a large economic impact. As a consequence, considerable research attention is now directed to understand better the software development process. The objective of this paper is to provide an experimental evaluation of the applicability, universality and accuracy of some algorithmic software cost estimating models (COCOMO, TUCOMO, PUTNAM, COPMO, ESSE and Function Points). Data on nine Italian MIS (Management Information Systems) projects were collected and used to evaluate the performance of the models. The evaluation of the estimates was based on the Mean Magnitude Relative Error ( MRE) and Prediction at level 25% (PRED(25%)) criteria. Results indicated that the models provided interesting performances, better if recalibrated with local data.
Abstract:
In the context of software cost estimation, system size is widely taken as a main driver of system development effort. But, other structural design properties, such as coupling, cohesion, and complexity, have been suggested as additional cost factors. In this paper, using effort data from an object-oriented development project, we empirically investigate the relationship between class size and the development effort for a class and what additional impact structural properties such as class coupling have on effort. This paper proposes a practical, repeatable, and accurate analysis procedure to investigate relationships between structural properties and development effort. This is particularly important as it is necessary, as for any empirical study, to be able to replicate the analysis reported here. More specifically, we use Poisson regression and regression trees to build cost prediction models from size and design measures and use these models to predict system development effort. We also investigate a recently suggested technique to combine regression trees with regression analysis which aims at building more accurate models. Results indicate that fairly accurate predictions of class effort can be made based on simple measures of the class interface size alone (mean MREs below 30 percent). Effort predictions at the system level are even more accurate as, using Bootstrapping, the estimated 95 percent confidence interval for MREs is 3 to 23 percent. But, more sophisticated coupling and cohesion measures do not help to improve these predictions to a degree that would be practically significant. However, the use of hybrid models combining Poisson regression and CART regression trees clearly improves the accuracy of the models as compared to using Poisson regression alone.
Abstract:
Delivering a soware product on time, within budget, and to an agreed level of quality is a critical concern for many software organizations. Underestimating software costs can have detrimental effects on the quality of the delivered software and thus on a company's business reputation and competitiveness. On the other hand, overestimation of the software cost can result in missed opportunities to funds in other projects. In response to industry demand, a myriad of estimation techniques has been proposed during the last three decades. in order to assess the suitability of a technique from a diverse selection, its performance and relative merits must be compared.
The current study replicates a comprehensive comparison of common estimation techniques within different organizational contexts, using data from the European Space Agency. Our study is motivated by the challenge to assess the feasibility of using multi-organization data to build cost models and the benefits gained from company-specific data collection. using the European Space Agency data set, we investigated a yet unexplored application domain, including military and space projects. The results showed that traditional techniques names, ordinary least-squares regression and analysis of variance outperformed analogy-based estimation and regression trees. Consistent with the results of the replicated study no significant difference was found in accuracy between estimates derived from company-specific data and estiamtes derived from multi-organizational data.
Abstract:
This paper investigates two essential questions related to data-driven, sofware cost modeling: (1) What modeling techniques are likely to yield more accurate resutls when using typical software development cost data? and (2) What are the benefits and drawbacks of using organization-secific data as compared to multi-organization databases? The former question is important in guiding software cost analysts in their choice of the right type of modeling technique, if at all ossible. In order to address this issue, we assess and compare a seletion of common cost modeling techniques fulfilling a number of important criteria using a large multi-organizational database in the business application domain. Namely, these are: ordinary least squares regression, stepwise ANOVA, CART, and analogy. The latter question is important in order to assess the feasibility of using multi-organization cost databases to build cost models and teh benfits gained from local, company-specific data collection and modeling. As a large subset of th data in the multi-compary database came from one organization , we were able to investigate the issue by comaring organization-specifc models with models based on multi-organization data. Results show that the performance of the modeling techniques considererd were not significantly different with the exception of the analyogy-based models which appear to be less accurate. Surprisingly, when using standard cost factors (e.g., COCOMO-like factors, Function Points), organization specific models did not yield better results than generic, multi-organiztaion models.
Notes: JO
Abstract:
Current cost estimation techniques have a number of drawbacks. For example, developing algorithmic models requires extensive past project data. Also, off-the-shelf models have been found to be difficult to calibrate but inaccurate without calibration. Informal approaches based on experienced estimators depend on estimatorsÕ availability and are not easily repeatable, as well as not being much more accurate than algorithmic techniques. In this paper we present a method for cost estimation that combines aspects of algorithmic and experiential approaches (referred to as COBRA, COst estimation, Benchmarking, and Risk Assessment). We find through a case study that cost estimates using COBRA show an average ARE of 0.09, and show that the results are easily usable for benchmarking and risk assessment purposes.
Notes: JO
Abstract:
To date many software engineering cost models have been developed to predict the cost, schedule, and quality of the software under development. But, the rapidly changing nature of software development has made it extremely difficult to develop empirical models that continue to yield high prediction accuracies. Software development costs continue to increase and practitioners continually express their concerns over their inability to accurately predict the costs involved. Thus, one of the most important objectives of the software engineering community has been to develop useful models that constructively explain the software development life-cycle and accurately predict the cost of developing a software product. To that end, many parametric software estimation models have evolved in the last two decades [25], [17], [26], [15], [28], [1], [2], [33], [7], [10], [22], [23]. Almost all of the above mentioned parametric models have been empirically calibrated to actual data from completed software projects. The most commonly used technique for empirical calibration has been the popular classical multiple regression approach. As discussed in this paper, the multiple regression approach imposes a few assumptions frequently violated by software engineering datasets. The source data is also generally imprecise in reporting size, effort, and cost-driver ratings, particularly across different organizations. This results in the development of inaccurate empirical models that don't perform very well when used for prediction. This paper illustrates the problems faced by the multiple regression approach during the calibration of one of the popular software engineering cost models, COCOMO II. It describes the use of a pragmatic 10 percent weighted average approach that was used for first publicly available calibrated version [6]. It then moves on to show how a more sophisticated Bayesian approach can be used to alleviate some of the problems faced by multiple regression. It compares and contrasts the two empirical approaches, and concludes the Bayesian approach was better and more robust than the multiple regression approach.
Bayesian analysis is a well-defined and rigorous process of inductive reasoning that has been used in many scientific disciplines reader can refer to [11], [35], [3] for a broader understanding of the Bayesian Analysis approach). A distinctive feature of the Bayesian approach is that it permits the investigator to use both sample (data) and prior (expert-judgment) information in a logically consistent manner in making inferences. This is done by using Bayes' theorem to produce a `postdata' or posterior distribution for the model parameters. Using Bayes' theorem, prior (or initial) values are transformed to postdata views. This transformation can be viewed a learning process. The posterior distribution is determined by the variances of the prior and sample information. If the variance of prior information is smaller than the variance of the sampling information, then a higher weight is assigned to the prior information. the other hand, if the variance of the sample information is smaller than the variance of the prior information, then a higher weight is assigned to the sample information causing the posterior estimate to be closer to the sample information. The Bayesian approach discussed in this paper enables stronger solutions to one of the biggest problems faced by the software engineering community: the challenge of making good decisions using data that is usually scarce and incomplete. We note that the predictive performance of the Bayesian approach (i.e., within 30 percent of the actuals 75 percent of the time) is significantly better than that of the previous multiple regression approach (i.e., within 30 percent of the actuals only 52 percent of the time) on our latest sample of 161 project datapoints.
Abstract:
Process improvement in manufacturing sector traditionally starts with calculation of Cost of Quality. Cost of Quality calculation enables identification of the weakest links of the process that requires immediate attention, prioritization of improvement tasks and establishment of a baseline for these tasks. In the software field, process improvement initiatives usually, leave out the cost of quality calculations. In this paper, we report a case study on the utilization of Cost of Software Quality as an initiator for software process improvement in a public software development organization.
Abstract:
Over the past decade, the term "software quality" has been widely used. In many instances the term has been loosely used in relation to process and product. This has created considerable confusion and diverted the industry from its primary goal - improving the quality of the products of the various phases of software development. While some attention has been paid to high-level quality attributes, little has been devoted to the systematic study of tangible product properties and their influence on high-level quality attributes. Today the dominant modus operandi for software development is heavily process-oriented. This rests on the widely held belief that you need a quality process to produce a quality product. The flaw in this approach is that the emphasis on process usually comes at the expense of constructing, refining, and using adequate product quality models. The fundamental axiom of software product quality is: a product's tangible internal characteristics or properties determine its external quality attributes. Developers must build these internal properties into a product in order for it to exhibit the desired external quality attributes. A product quality model, therefore, must comprehensively identify the tangible (measurable and/or assessable) internal product characteristics that have the most significant effect on external quality attributes. I suggest a framework for the construction and use of practical, testable quality models for requirements, design, and implementation. Such information may be used directly to build, compare, and assess better quality software products.
Abstract:
Understanding how to implement software process improvement (SPI) successfully is arguably the most challenging issue facing the SPI field today. The SPI literature contains many case studies of successful companies and descriptions of their SPI programs. However, the research efforts to date are limited and inconclusive and without adequate theoretical and psychometric justification. This paper extends and integrates models from prior research by performing an empirical investigation of the key factors for success in SPI. A quantitative survey of 120 software organizations was designed to test the conceptual model and hypotheses of the study. The results indicate that success depends critically on six organizational factors, which explained more than 50 percent of the variance in the outcome variable. The main contribution of the paper is to increase the understanding of the influence of organizational issues by empirically showing that they are at least as important as technology for succeeding with SPI and, thus, to provide researchers and practitioners with important new insights regarding the critical factors of success in SPI.
Abstract:
We consider a set of programs a family when it pays to look at their common aspects before looking at their differences. For commercial software developers the implications are twofold: First, making rational decisions about product-line processes and products requires the ability to answer the question: "Does it pay?" Second, whether or not something pays is ultimately a business (rather than software engineering) question. In short, making sound software engineering decisions requires understanding the business implications of those decisions, and vice versa.
This paper describes work in progress to develop a product-line process model and common value metric that adequately link product value drivers (what it pays) with the software engineering decisions that affect those drivers. In it, we describe a systematic approach to quantifying the return for both product and process improvements based on common software engineering principles and a common value metric, customer value.
Abstract:
A number of issues are covered in this paper. Chief among them is the need for greater discipline in understanding the economic benefits of software reuse within the context of a broader business strategy. While traditional methods fail to account for growth opportunities and flexibility generated by investments in reuse, the introduction of option pricing theory can greatly enhance the design and evaluation of software reuse projects. Similarly, the disciplines of business strategy hold promise to help to fill the void of ``strategic context'' within which reuse investment happens.
Abstract:
In this article, we show how prediction models are used to improve delivered quality. We further show that if you can anticipate and plan for the factors that affect quality, you can leverage quality management activities to improve the entire development effort.
Abstract:
Estimation of project development effort is most often performed by expert judgment rather than by using an empirically derived model (although such may be used by the expert to assist their decision). One question that can be asked about these estimates is how stable are they with respect to characteristics of the development process and product? This stability can be assessed in relation to the degree to which the project has advanced over time, the type of module for which the estimate is being made, and the characteristics of that module. In this paper we examine a set of expert-derived estimates for the effort required to develop a collection of modules from a large health-care system. Statistical tests are used to identify relationships between the type (screen or report) and characteristics of modules and the likelihood of the associated development effort being underestimated, approximately correct, or over-estimated. Distinct relationships are found that suggest that the estimation process being examined was not unbiased to such characteristics. This is a potentially useful finding in that it provides an opportunity for estimators to improve their prediction performance.
Abstract:
This study draws upon theories of task interdependence and organizational inertia to analyze the effect of quality improvement on infrastructure activity costs in software development. Although increasing evidence indicates that quality improvement reduces software development costs, the impact on infrastructure activities is not known. Infrastructure activities include services like computer operations, data integration, and configuration management that support software development. Because infrastructure costs represent a substantial portion of firms' information technology budgets, it is important to identify innovations that yield significant cost savings in infrastructure activities. We evaluate quality and cost data collected in nine infrastructure activity centers over 10 years of product development in a major software firm undergoing a quality transformation. Findings indicate that infrastructure activities do benefit from quality improvement. The greatest marginal cost savings are realized in infra structure activities that are highly interdependent with development and that occur later in the software development life cycle. Organizational inertia influences the rapidity with which the infrastructure activities benefit from higher product quality, especially for the more specialized activities. Finally, our findings suggest that although the savings in infrastructure from quality improvement are substantial, there are diminishing returns to quality improvement in infrastructure activities.
Abstract:
The information technology industry is characterized by rapid innovation and intense competition. To survive, IT firms must develop high quality software products on time and at low cost. A key issue is whether high levels of quality can be achieved without adversely impacting cycle time and effort. The relationship between process maturity, quality, cycle time, and effort is investigated for the development of 30 software products by a major IT firm. It is found that higher levels of process maturity as assessed by the Software Engineering Institute's Capability Maturity model are associated with higher product quality, but also with increases in development effort. Findings indicate that the reductions in cycle time and effort due to improved quality outweigh the increases from achieve higher levels of process maturity. This, the net effects of process maturity is reduced cycle time and development effort.
Abstract:
Software size is a fundamental product measure that can be used for assessment, prediction, and improvement purposes. However, existing software size measures, such as Function Points, do not address the underlying problem complexity of software systems adequately. This can result in disproportional measures of software size for different types of systems. We propose a Vector Size Measure (VSM) that incorporates both functionality and problem complexity in a balanced and orthogonal manner. VSM is used as the input to a Vector Prediction Model (VPM) which can be used to estimate development effort early in the software life cycle. We theoretically validate the approach against a formal framework. We also empirically validate the approach with a pilot study. The results indicate that the approach provides a mechanism to measure the size of software systems, classify software systems, and estimate development effort early in the software life cycle to within +/-20 percent across a range of application types.
Abstract:
Estimating the amount of effort required for developing an information system is an important project management concern. In recent years, a number of studies have used neural networks in various stages of software development. This study compares the prediction performance of multilayer perceptron and radial basis function neural networks to that of regression analysis. The results of the study indicate that when a combined third generation and fourth generation languages data set were used, the neural network produced improved performance over conventional regression analysis in terms of mean absolute percentage error.
Abstract:
The reason for this strong emphasis on software cost estimation is that it provides the vital link between the general cocepts and techniques of economic analysis and the particular world of software engineering. Software cost estimation techniques also provides an essential part of the foundation for good software management.Ê
Abstract:
Cost of Quality (CoQ) is a concept which has been used extensively in manufacturing and service industries to justify, identify, and promote process improvement initiatives. This paper looks at the limited use thus far of CoQ in software development (Cost of Software Quality - CoSQ). Available data indicates very high CoSQ for the majority of software producers, but reveals the possibility of significant savings from ongoing process improvement programs. The benefits of CoSQ are discussed and one of them is illustrated using the results of dynamic process modeling.
Abstract:
Project LEAP (lightweight, empirical, antimeasurement dysfunction, and portable toolkit) is investigating tools and methods to support low-cost, empirically based software developer improvement. LEAP contains tools to simplify the collection of size and effort data. The collected data serves as input to a set of estimation tools. These tools can produce over a dozen analytical estimates of the effort required for a new project given an estimate of its size. To do this, they use various estimation methods such as linear, logarithmic, or exponential regressions. During project planning, the developer can review these estimates and select one of them or substitute a guesstimate based on his or her experience. The authors' study provides evidence that guesstimates, when informed by low-cost analytical methods, might be the most accurate method.
Abstract:
Traditionally, software professionals are requested to provide minimum-maximum intervals to indicate the uncertainty of their effort estimates. In this paper, we claim that the traditional request is not optimal and leads to overoptimistic views about the level of estimation uncertainty. Instead, we propose that it is better to frame the request for uncertainty assessment: "How likely is it that the actual effort will be more than/less than X?Ó Our claim is based on the results of a previously reported experiment and field studies in two companies. The two software companies were instructed to apply the traditional and our alternative, framing on random samples of their projects. In total, we collected information about 47 projects applying the traditional framing and 23 projects applying the alternative framing.
Abstract:
This paper provides an extensive review of studies related to expert estimation of software development effort. The main goal and contribution of the review is to support the research on expert estimation, e.g., to ease other researcher's search for relevant expert estimation studies. In addition, we provide software practitioners with useful estimation guidelines, based on the research-based knowledge of expert estimation processes. The review results suggest that expert estimation is the most frequently applied estimation strategy for software projects, that there is no substantial evidence in favour of use of estimation models, and that there are situations where we can expect expert estimates to be more accurate than formal estimation models. The following 12 expert estimation "best practice" guidelines are evaluated through the review: (1) evaluate estimation accuracy, but avoid high evaluation pressure; (2) avoid conflicting estimation goals; (3) ask the estimators to justify and criticize their estimates; (4) avoid irrelevant and unreliable estimation information; (5) use documented data from previous development tasks; (6) find estimation experts with relevant domain background and good estimation records; (7) Estimate top-down and bottom-up, independently of each other; (8) use estimation checklists; (9) combine estimates from different experts and estimation strategies; (10) assess the uncertainty of the estimate; (11) provide feedback on estimation accuracy and development task relations; and, (12) provide estimation training opportunities. We found supporting evidence for all 12 estimation principles, and provide suggestions on how to implement them in software organizations.
Abstract:
The uncertainty of a software development effort estimate can be indicated through a prediction interval (PI), i.e., the estimated minimum and maximum effort corresponding to a specific confidence level. For example, a project manager may be "90% confident" or believe that is it "very likely" that the effort required to complete a project will be between 8000 and 12,000 work-hours. This paper describes results from four studies (Studies AÐD) on human judgement (expert) based PIs of software development effort. Study A examines the accuracy of the PIs in real software projects. The results suggest that the PIs were generally much too narrow to reflect the chosen level of confidence, i.e., that there was a strong over-confidence. Studies BÐD try to understand the reasons for the observed over-confidence. Study B examines the possibility that the over-confidence is related to type of experience or estimation process. Study C examines the possibility that the concept of confidence level is difficult to interpret for software estimators. Finally, Study D examines the possibility that there are unfortunate feedback mechanisms that reward over-confidence.
Abstract:
Expert estimation of software development effort may follow top-down or bottom-up strategies, i.e. the total effort estimate may be based on properties of the project as a whole and distributed over project activities (top-down) or calculated as the sum of the project activity estimates (bottom-up). The explorative study reported in this paper examines differences between these two strategies based on measurement and video recording of the discussions of seven estimation teams. Each estimation team applied a top-down estimation strategy on one project and a bottom-up estimation strategy on another. The main contribution of the study is the observation that the recall of very similar previously completed projects seemed to be a pre-condition for accurate top-down strategy based estimates, i.e. the abilities of the software estimators to transfer estimation experience from less similar projects was poor. This suggests that software companies should apply the bottom-up strategy unless the estimators have experience from, or access to, very similar projects.
Abstract:
Estimation by analogy is, simplified, the process of finding one or more projects that are similar to the one to be estimated and then derive the estimate from the values of these projects. If the selected projects have an unusual high or low productivity, then we should adjust the estimates toward productivity values of more average projects. The size of the adjustments depends on the expected accuracy of the estimation model. This paper evaluates one adjustment approach, based on the findings made by Sir Francis Galton in the late 1800s regarding the statistical phenomenon "regression toward the mean" (RTM). We evaluate this approach on several data sets and find indications that it improves the estimation accuracy. Surprisingly, current analogy based effort estimation models do not, as far as we know, include adjustments related to extreme analogues and inaccurate estimation models. An analysis of several industrial software development and maintenance projects indicates that the effort estimates provided by software professionals, i.e., expert estimates, to some extent are RTM-adjusted. A student experiment confirms this finding, but also indicates a rather large variance in how well the need for RTM-adjustments is understood among software developers.
Abstract:
The paper presents a methodology for estimation of software size and effort at early stages of software development. The research concentrates on the size estimation problem, which seems to be the weakest element of cost estimation. The methodology concerns the object-oriented technology and the Java language.
Authors review current techniques of size and cost estimation to identify their strengths and weaknesses. The paper describes statistical characteristics of class and method sizes for programs written in Java. The analysis of nearly one million lines of code lead to the conclusion that the average class size and the average method size is independent from application size. This feature is useful in calculating the final application size if the number of classes is known or could be estimated during software development.
The paper contains definitions of three simple models of size estimation that are based on class and method sizes. The presented approach may be easily applied as it uses data that is typically produced during early stages of software development. The experimental model was theoretically verified and analysed. Further, an independent set of applications written in Java was used to verify the correctness of the acquired equations and models. Statistical characteristics were acquired and analysed with the use of a dedicated tool that was implemented as a part of the research.
Abstract:
If you are a software developer, manager, or maintainer, quality is often on your mind. But what do you really mean by software quality? Is your definition adequate? Is the software you produce better or worse than you would like it to be? In this special issue, we put software quality on trial, examining both the definition and evaluation of our software products and processes.
Abstract:
Cost of software quality (CoSQ) is an accounting technique that is useful to enable our understanding of the economic trade-offs involved in delivering good-quality software. Commonly used in manufacturing, its adaptation to software offers the promise of preventing poor quality but, unfortunately, has seen little use to date. This article discusses the rationale and context for using CoSQ, then defines a basic CoSQ model that differentiates the costs involved with handling nonconformances due to a lack of quality, appraisal efforts performed for the achievement of acceptable quality, and efforts to prevent poor quality from occurring.
Abstract:
We examine the relationship between life-cycle productivity and conformance quality in software products. The effects of product size, personnel capability, software process, usage of tools, and higher front-end investments on productivity and conformance quality were analyzed to derive managerial implications based on primary data collected on commercial software projects from a leading vendor. Our key findings are as follows. First, our results provide evidence for significant increases in life-cycle productivity from improved conformance quality in software products shipped to the customers. Given that the expenditure on computer software has been growing over the last few decades, empirical evidence for cost savings through quality improvement is a significant contribution to the literature. Second, our study identifies several quality drivers in software products. Our findings indicate that higher personnel capability, deployment of resources in initial stages of product development (especially design) and improvements in software development process factors are associated with higher quality products.
Abstract:
This paper studied the effect of people factors in determining costs and quality (defined based on the number of unique defects reported) in packaged software products. The analysis indicates that personnel capability of the team is significantly associated with lower number of defects and enhanced productivity in packaged software products. Also, teams with higher experience in specific application domain of the product exhibit lower number of defects and thus incur lower maintenance costs. For managers of packaged software projects, results of the three empirical models provide several insights for composition of the teams for a successful packaged software project. First, results indicate the significant effect of the personnel capability of the team. This shows that in the package software environment, just as in more traditional environments, using a highly capable team helps in reducing product defects, and enhancing both development and maintenance productivity. Second, using personnel with a higher experience in a specific software domain aids in delivering products with less number of defects.
Abstract:
This paper offers an extrapolation of the manufacturing and service industries' Cost of Quality Model to the business of software development. The intent is to provide a theoretical account of the changing quality cost structure as a function of a maturing softwaer development process. Thus, the trends in expenditures due to the four major quality cost categories - appraisal, prevention, internal failures, and external failures - are presentd over the five levels of software process maturity, according to the Software Engineering Institute's (SEI's) Capability Maturity Model for Software (CMM). The Software Cost of Quality Model conservatively proposes tha th etotal cost of quality, expressed as a percentage of the cost of development, can be decreased approximately two-thirds as process maturity grows from level 1 to Level 5 of the SEI's CMM.
Notes: JO
Abstract:
Budgetary constraints are placing increasing pressure on project managers to effectively estimate development effort requirements at the earliest opportunity. With the rising impact of automation on commercial software development the attention of researchers developing effort estimation models has recently been focused on functional representations of systems, in response to the assertion that development effort is a function of specification content [1]. A number of such models exist -several, however, have received almost no research or industry attention. Project managers wishing to implement a functional assessment and estimation programme are therefore unlikely to be aware of the various methods or how they compare. This paper therefore attempts to provide this information, as well as forming a basis for the development and improvement of new methods.
Abstract:
We set out to answer a question we were asked by software project management: how much effort remains to be spent on a specific software project and how will that effort be distributed over time? To answer this question we propose a model based on the concept that each modification to software may cause repairs at some later time and investigate its theoretical properties and application to several projects in Avaya to predict and plan development resource allocation. Our model presents a novel unified framework to investigate and predict effort, schedule, and defects of a software project. The results of applying the model confirm a fundamental relationship between the new feature and defect repair changes and demonstrate its predictive properties.
Abstract:
To have general validity, empirical results must converge. To be credible, an experimental science must understand the limitations and be able to explain the disagreements of empirical results. We describe an experiment to replicate previous studies which claim that estimation by analogy outperforms regression models. In the experiment, 68 experienced practitioners each estimated a project from a dataset of 48 industrial COTS projects. We applied two treatments, an analogy tool and a regression model, and we used the estimating performance when aided by the historical data as the control. We found that our results do not converge with previous results. The reason is that previous studies have used other datasets and partially different data analysis methods, and last but not least, the tools have been validated in isolation from the tool users. This implies that the results are sensitive to the experimental design: the characteristics of the dataset, the norms for removing outliers and other data points from the original dataset, the test metrics, significance levels, and the use of human subjects and their level of expertise. Thus, neither our results nor previous results are robust enough to claim any general validity.
Abstract:
This paper outlines a four step effort estimation study and focuses on the first and second step. The four steps are formulated to successively introduce a more formal effort experience base. The objective of the study is to evaluate the needed formalism to improve effort estimation and to study different approaches to record and reuse experiences from effort planning in software projects. In the first step (including seven projects), the objective is to compare estimation of effort based on a rough figure (indicating approximate size of the projects) with an informal experience base. The objective of the second step is on reuse of experiences from an effort experience base, where the outcomes of seven previous projects have been stored. Seven new projects are planned based on the previous experiences.
The plans are, after project completion, compared with the initial plans and with the data from six out of the seven new projects to plan the seventh. It is clear from the studies that effort estimation is difficult, and that the mean estimation error is in the range of 14-19 percent independent of the approach used. Furthermore, it is concluded that the best estimates are obtained when the projects use the previous experience and complement this information with their own thoughts and opinions. Finally, it is concluded that data collection is not enough in itself, the data collected must be processed, i.e. interpreted, generalised and synthesised into a reusable form.
Abstract:
In object-oriented analysis, use case models describe the functional requirements of a future software system. Sizing the system can be done by measuring the size or complexity of the use cases in the use case model. The size can then serve as input to a cost estimation method or model, in order to compute an early estimate of cost and effort.
Estimating software with use cases is still in the early stages. This thesis describes a software sizing and cost estimation method based on use cases, called the 'Use Case Points Method'. The method was created several years ago, but is not well known. One of the reasons may be that the method is best used with well-written use cases at a suitable level of functional detail. Unfortunately, use case writing is not standardized, so there are many di erent writing styles. This thesis describes how it is possible to apply the use case points method for estimating object-oriented software, even if the use cases are not written out in full. The work also shows how use cases can be sized in alternative ways, and how to best write use cases for estimation purposes. An extension of the method providing simpler counting rules is proposed.
Two case studies have been conducted in a major software company, and several student's projects have been studied in order to investigate the general usefulness of the method and its extension. The results have been compared to results obtained earlier using the method in a di erent company. The investigations show that the use case points method works well for di erent types of software.
Data from the various projects have also been used as input to two commercial cost estimation tools that attempt to estimate object-oriented projects with use cases. The goal was to select a cost estimation method or tool for a speci c software company. The ndings indicate that there is no obvious gain in investing in expensive commercial tools for estimating object-oriented software. iii
Abstract:
The staff resources or effort required for a software project are notoriously dificult to estimate in advance. To date most work has focused upon algorithmic cost models such as COCOMO and Function Points. These can suffer from the disadvantage of the need to calibrate the model to each individual measurement environment coupled with very variable accuracy levels even after calibration. An alternative approach is to use analogy for estimation. We demonstrate that this method has considerable promise in that we show it to out perform traditional algorithmic methods for six different datasets. A disadvantage of estimation by analogy is that it requires a considerable amount of computation. This paper describes an automated environment known as ANGEL that supports the collection, storage and identification of the most analogous projects in order to estimate the effort for a new project. ANGEL is based upon the minimisation of Euclidean distance in n-dimensional space. The software is flexible and can deal with differing datasets both in terms of the number of observations (projects) and in the variables collected. Our analogy approach is evaluated with six distinct datasets drawn from a range of different environments and is found out perform other methods. It is widely accepted that effective software effort estimation demands more than one technique. We have shown that estimating by analogy is a candidate technique and that with the aid of an automated environment is an eminently practical technique.
Abstract:
Discussions of software quality typically focus on the development process or the characteristics of the software product. The third level of quality - the outcome of software development - is usually neglected, although this is perhaps of greatest interest to business management. Outcome depends on how the product is used and determines the business value obtained from the development project. I conducted a study that investigated how a number of Australian organizations make their IT investment decisions. My research focused on how organizations evaluated individual projects, both at the justification stage and after implementation. I also examined how "effectiveness" was interpreted (that is, what was considered to be good business), the extent to which nonfinancial benefits influenced investment decisions, and the type of measures that were considered useful. From this work I developed a comprehensive framework of benefits and measures, and I validated the framework using the actual justifications of 25 projects. Thus, the framework provides a useful model for the analysis and measurement of the multiple dimensions of value that IT can bring to a business.
Abstract:
In most software development organizations, there is seldom a one-to-one mapping between software developers and development tasks. It is frequently necessary to concurrently assign individuals to multiple tasks and to assign more than one individual to work cooperatively on a single task. A principal goal in making such assignments should be to minimize the effort required to complete each task. But what impact does the manner in which developers are assigned to tasks have on the effort requirements? This paper identifies four task assignment factors: team size, concurrency, intensity, and fragmentation. These four factors are shown to improve the predictive ability of the well-known Intermediate COCOMO cost estimation model. A parsimonious effort estimation model is also derived that utilizes a subset of the task assignment factors and Unadjusted Function Points. For the data examined, this parsimonious model is shown to have goodness of fit and quality of estimation superior to that of the COCOMO model, while utilizing fewer cost factors.
Notes: JO
Notes: JO
Abstract:
The management of organizational change is an essential element of successful software process improvement efforts. This paper describes ten factors that affect organizational change in software process improvement initiatives based on the Capability Maturity Model or the ISO 9000 quality standards. It also assesses the relative importance of these factors and compares the findings with the results of previous research into organizational change in software process improvement. The paper is based on an analysis of published experience reports and case studies of 56 software organizations that have implemented an ISO 9000 quality system or that have conducted a CMM-based process improvement initiative.
Abstract:
The concept of information hiding modularity is a cornerstone of modern software design thought, but its formulation remains casual and its emphasis on changeability is imperfectly related to the goal of creating value in a given context. We need better models of the structure and value of information hiding, for both their explanatory power and prescriptive utility. We evaluate the potential of a new theory-developed to account for the influence of modularity on the evolution of the computer industry-to inform software design. The theory uses design structure matrices to model designs and real options techniques to value them. To test the potential utility of the theory for software we represent a model software system in its terms-Parnas's KWIC-and evaluate the results. We contribute an extension to design structure matrices and show that the options results are consistent with Parnas's conclusions. Our results suggest that such a theory does have potential to help inform software design.
Abstract:
Many software engineering principles and concepts that are critical to reasoning about problems in software design (of which software architecture is an important special case) remain ad hoc, idiosyncratic and poorly integrated. I argue that this is due to our lack of a clean theory about how to make software design decisions. In this paper I propose that we should view software design as a process of deciding how to make irreversible capital investment in software assets of uncertain value, and that financial options theory provides a firm, unifying, simplifying and well developed basis for such decision-making. To support this view, I interpret software architecture and other related concepts in options theoretic terms.
Abstract:
Debate using relevant, domain-specific quality terms is a mark of mature theory and expertise in a particular area. Wine connoisseurs and movie critics use domain-specific quality terms that even novices are at least partly familiar with. What about software? Is software engineering mature enough so that software designers can use quality terms in discussions with their peers? Do inspectors of software artifacts evaluate them in terms of quality? Is the final software product advertised in such terms? Do customers understand the quality-based advantages of competing products? Most software engineers agree that there is too much variety in quality terms. Quality models such as Software Quality Metrics and Goal Question Metric and standards such as the IEEE Quality Metrics Methodology and ISO 9126 provide a quality terminology, but they tend to be too rigid or abstract, and of course they cannot guarantee that their terminology will be learned and used. This prevents software engineers from understanding the role of quality in practice: how to state quality goals, how to justify alternative solutions in quality terms, and how to evaluate the achievement of quality in alternative solutions. How do we solve this problem? In theory, the solution is easy. We need an appropriate quality model, a software-development method that uses this quality model, a supporting tool, and adequate training in the use of the method and tool. In practice, we all know how difficult this is. I propose to use a consistent quality structure for designing and inspecting software artifacts. This structure is based on a quality model that I call SQM synthesis because of my numerous modifications to the original Software Quality Metrics model.
Abstract:
Software engineering was been around for many years. The industry has not matured as fast as it grew. Software engineering accounts for 12 million jobs and one trillion dollars of sales in the US alone. One of the many lacking qualities in software engineering is the ability to estimate the size of software projects. Software estimation techniques have also been around for quite a few years. However, the quality of these techniques and models were low. It is the purpose of this paper to provide insight into the newest estimation techniques that can improve quality and productivity of software projects. It takes more the an estimation technique to improve the quality and productivity of software. It requires a formal change in the existing process and a change in the existing though process. As a last topic, some existing CASE tools that assist in code estimation are examined.
Abstract:
This work is based on a retrospective analysis of a renewal process applied to a very aged, highly degraded software system. Some aspects of the results are generalized to define a method for determining the technical and economic qualities of the components of a software system. The work also presents a decision model for identifying the most suitable renewal process to be applied, on the basis of the quality of the components and the aims of the renewal process. With the model presented, decision-making on the renewal process is specialized to each component of the software system, thus ensuring greater benefits from the process as a whole. The same model can be used to monitor the decay of a software system quality and thus avoid the need to renew it using the most costly renewal processes.
Abstract:
There are three distinct approaches to certifying the quality of software: accrediting personnel, certifying the development organization, and assessing the "goodness" of the software. These approaches, and hybrids thereof, are described, and criteria are given to determine which approach is best, depending on the software that needs to be certified.
Abstract:
The objective of this paper is to investigate the pertinent question whether multi-organizational data is valuable for software project cost estimation. Local, company-specific data is widely believed to provide a better basis for accurate estimates. On the other hand, multi-organizational databases provide an opportunity for fast data accumulation and shared information benefits. Therefore, this paper trades off the potential advantages and drawbacks of using local data as compared to multi-organizational data.
Motivated by the results from previous investigations, we further analyzed a large cost database from Finland that collects standard cost factors and includes information on six individual companies. Each of these companies provided data for more than ten projects. This information was used to compare the accuracy between company-specific (local) and company-external (global) cost models. Consistent with some of the previous investigations in this area, the general trends in the results are surprising. They show that company-specific models seem not to yield better results than the company-external models. Our results are based on applying two standard statistical estimation methods (OLS-regression, Analysis of Variance) and Analogy-based estimation.
Abstract:
The Function Point Method, estimation by analogy, and algorithmic modeling are three of the most commonly applied methods used to estimate the costs and worker hours needed for a software development project. These methods, however, require a deep and wide expertise in particular areas and may still result in unacceptable discrepancies between the estimated costs and the actual costs. This paper presents a framework for a top-down cost estimation method (TCE). The method is based on the assumption that different types of software have different intrinsic complexities. We expect that this method will produce easier, faster, and more accurate estimations in the early stages of a software project.
Abstract:
This paper describes statistical approaches for analyzing software cost and quality data of Software Development Center of Hitachi (SDCH) in recent several years. We have got some highly fitted cost and quality models with r = 0.837 - 0.975 multiple correlation. The models may be used for cost and quality estimation and also productivity and quality evaluation.
Abstract:
Software engineers often espouse the importance of using abstraction and encapsulation in developing software components. They advocate the “layering” of new components on top of existing components, using only information about the functionality and interfaces provided by the existing components. This layering approach is in contrast to a “direct implementation” of new components, utilizing unencapsulated access to the representation data structures and code present in the existing components. By increasing the reuse of existing components, the layering approach intuitively should result in reduced development costs, and in increased quality for the new components. However, there is no empirical evidence that indicates whether the layering approach improves developer productivity or component quality.
We discuss three controlled experiments designed to gather such empirical evidence. The results support the contention that layering significantly reduces the effort required to build new components. Furthermore, the quality of the components, in terms of the number of defects introduced during their development, is at least as good using the layered approach.
Experiments such as these illustrate a number of interesting and important issues in statistical analysis. We discuss these issues because, in our experience, they are not well-known to software engineers.
Last Modified: June 29, 2005
Comments should be sent to
Richard Upchurch (rupchurch@umassd.edu)
Computer and Information Science Department