Validation of a Standard- and Metric-Based Software Quality Model

Sammanfattning: The automatic assessment of software quality using standard- and metric-based Software Quality Models has numerous advantages. For example, it is less error-prone, less time consuming, and neutral to human judgments compared to manual approaches. Yet, to rely on the results, the Software Quality Models need to be validated, i.e., experiments should show that the measured quality and the experienced quality correlate. Such validations are rare and the few that exist are hard to generalize since they are ambiguously defined.In detail, to be able to generalize from a validation of a standard- and metric-based Software Quality Model, the following prerequisites need to be fulfilled: First, the definitions of the software quality metrics need to be unambiguous. Second, a formal meta-model for the software assessed and a mapping from the (language specific) software elements to the elements of a formal meta-model need to be defined. Finally, the definition of the Software Quality Model needs to be exact. The prerequisites for generalization stated above are not fulfilled in practice. This is shown in the first part of this thesis, which is at first based on practical observations and then on experiments.In order to address the above issues, we define a common meta-model as well as a mapping from language specific software elements to meta-model elements. This meta-model is then the basis for metrics definitions; we redefine about 25 well-known object-oriented metrics using our meta-model. Using these metrics, we introduce a Software Quality Model based on ISO 9126. It defines quantitatively the relations between the metrics and the quality factors and criteria. Fulfilling the prerequisite unambiguous experiment definition, as described in the second part of this thesis, allows us to validate the Software Quality Model in experiments and generalize from the results.In the third part, we describe the main experiment: Using multicollinearity analysis, we show (i) that some metrics suggested in literature are linearly dependent. Using univariate regression analysis, we show (ii) that a Software Quality Model for maintainability based on the linearly independent metrics correlates with the number of revisions in successful open source projects--later revisions have better maintainability than earlier ones. Using multivariate regression analysis, we show (iii) how these metrics correlate with the number of iterations in the projects. For (i), we analyze 157 different software projects with over 70,000 classes. For (ii) and (iii), we analyze over 300 versions of 11 well-known open source projects. As a side effect, we extracted the distribution and threshold values for the well-known object-oriented metrics defined on classes.Besides, we created tools and processes for monitoring software quality which may be used in a number of scenarios. In the development process, they allow early corrective actions. Under maintenance, they allow a precise assessment of required efforts of change and redevelopment. For project management, they allow to control if subcontractors or outsourced software developers meet the agreed quality goals. We show the feasibility of our tools and processes in a number of such practical cases.