Improving Risk Analysis Practices in Governmental Organizations

Sammanfattning: At the same time as our dependence on IT systems increases, the number of reports of problems caused by failures of critical IT systems has also increased. Today almost every system or service, e.g., water, power supply, transportation, is dependent on IT systems, and failure of these systems has serious and negative effects on society. In general, governmental organizations are responsible for delivery of these services to society. The increasing dependence on critical IT systems also makes them more and more complex. Risk analysis is an important activity for the development and operation of critical IT systems, but the increased complexity and size put additional requirements on the effectiveness of risk analysis methods. Risk analysis of technical systems has a long history in mechanical and electrical engineering. Even if a number of methods for risk analysis of technical systems exist, the failure behavior of information systems is typically very different from mechanical systems. Therefore, risk analysis of IT systems requires different risk analysis techniques, or at least adaptations of traditional approaches. The research objective of this thesis is to improve the analysis process of risks pertaining to IT systems in governmental organizations. In this thesis the improvements in risk analysis processes are addressed in two different ways. First, by understanding what types of methods are available for IT systems and how they can be improved. Second, by developing new effective and efficient risk analysis methods that can be useful to analyze IT systems in governmental organizations. In this thesis work, a systematic mapping study was carried out to understand existing methods and techniques used for analyzing IT systems. It found very few empirical research papers about the evaluation of existing risk analysis methods. The results of the mapping study suggest to empirically investigate risk analysis methods for analyzing IT systems to conclude which methods are more effective than others. Based on the results of the mapping study a case study was carried out to evaluate the effectiveness and efficiency of an existing risk analysis method, System Theoretic Process Analysis (STPA). Based on the results of the mapping study a controlled experiment was carried out to evaluate the effectiveness of risk analysis methods. The effectiveness of risk analysis methods was evaluated by counting the number of relevant and non-relevant risks identified by the experiment participants. The difficulty level of risk analysis methods and the experiment participants’ confidence about the identified risks were also investigated. The work presented in this thesis also presents a new risk analysis method, Perspective Based Risk Analysis (PBRA), that uses different perspectives while analyzing IT systems. A perspective is a point of view or a specific role adopted by risk analyst while doing risk analysis, i.e., system engineer, system tester, or system user. A case study was carried out to save historical information about IT incidents to be used later for risk analysis. This study investigates how difficult it is to find relevant risks from the available sources and the effort required to set up such a system. It also investigates how accurate the found risks are. It is believed that this could be an important aid in the process of building a database of occurred IT incidents that later can be used as an input to improve the risk analysis process. The presented research work in this thesis provides research about methods and tools for governmental organizations to improve their risk analysis and management practices. Moreover, the presented work in this thesis is based on solid empirical studies.