Towards Optimization of Anomaly Detection Using Autonomous Monitors in DevOps

Sammanfattning: Continuous practices including continuous integration, continuous testing, and continuous deployment are foundations of many software development initiatives. Another very popular industrial concept, DevOps, promotes automation, collaboration, and monitoring, to even more empower development processes. The scope of this thesis is on continuous monitoring and the data collected through continuous measurement in operations as it may carry very valuable details on the health of the software system. Aim: We aim to explore and improve existing solutions for managing monitoring data in operations, instantiated in the specific industry context. Specifically, we collaborated with a Swedish company responsible for ticket management and sales in public transportation to identify challenges in the information flow from operations to development and explore approaches for improved data management inspired by state-of-the-art machine learning (ML) solutions.Research approach: Our research activities span from practice to theory and from problem to solution domain, including problem conceptualization, solution design, instantiation, and empirical validation. This complies with the main principles of the design science paradigm mainly used to frame problem-driven studies aiming to improve specific areas of practice. Results: We present identified problem instances in the case company considering the general goal of better incorporating feedback from operations to development and corresponding solution design for reducing information overflow, e.g. alert flooding, by introducing a new element, a smart filter, in the feedback loop. Therefore, we propose a simpler version of the solution design based on ML decision rules as well as a more advanced deep learning (DL) alternative. We have implemented and partially evaluated the former solution design while we present the plan for implementation and optimization of the DL version of the smart filter, as a kind of autonomous monitor. Conclusion: We propose using a smart filter to tighten and improve feedback from operations to development. The smart filter utilizes operations data to discover anomalies and timely report alerts on strange and unusual system's behavior. Full-scale implementation and empirical evaluation of the smart filter based on the DL solution will be carried out in future work.