Observing the dynamic behaviour of large distributed systems to improve development and testing : an empirical study in software engineering

Sammanfattning: Knowledge about software systems' dynamics is a prerequisite for the successful design, testing and maintenance of viable products. This work has evolved a number of tools based on observation of software system dynamics in a commercial environment, resulting in a method and a toolbox that can be used by testers and maintainers to improve both the system and its test environment. The toolbox uses interceptors to observe the object interaction on the CORBA level during execution. With interceptors it is possible to intercept object communication without the need of the source code and with low impact on system performance. Intercepting a series of messages between the various objects can create an image of specific dynamic aspects of a running system. Observation can also be combined with simulation via active probing. Here active probing denote delays in communication and other simulated resource limitations that can be injected into the system for capacity testing purposes. The method conceptually supports plan-do-study-act promoted by Shewhart. The method is created to handle at least four different development activities: system tuning, testing, test evaluation, usage evaluation and increasing software understanding in general. System tuning can be activities such as performance enhancements or load balancing. The method also serves user profiling if it can run at a customer site. With coverage measurements, for example, how each internal function is covered during testing, one gets a measure of test quality and a way to specify goals for testing. With active probing, it will be possible to effect execution of a program. This can be used for system-robustness testing or as an oracle as how the system will react in different real-life situations. The need of a general understanding is documented with an interview series of software professionals. Yet another interview series with professionals using the tool shows how understanding can be enhanced. The method has been developed and evaluated in several case studies at different branches of ERICSSON AB in Linköping and Stockholm, Sweden. It is planned to become an integrated part of ERICSSON's O&M platform from 2004.