Model-based System Testing of Safety-Critical Embedded Software

Sammanfattning: System-level testing of safety-critical embedded systems is complex and costly. MBT has shown promising results in terms of fault detection effectiveness and efficiency of test generation and execution. However, the industrial adoption of MBT approaches is slow and limited to specific industries and domains. Moreover, the strengths and weaknesses of MBT in industrial settings need to be thoroughly evaluated to find an optimal testing strategy.The objective of this thesis is to adapt, compare, and evaluate the effectiveness and efficiency of MBT to help industrial practitioners in the testing of safety-critical embedded software. We have divided this objective into three subgoals. To achieve the first subgoal, we have explored multiple state-of-the- art MBT tools and evaluated the selected tool, GraphWalker (GW), in terms of modeling notations, generation algorithm, stopping conditions, and model completeness. To achieve the second subgoal, we have proposed a Model-Based Test scrIpt GenEration fRamework (TIGER), based on GW, to generate system-level test artifacts (i.e., test cases and test scripts). Based on the proposed framework, we implemented two test script generation tools for combinatorial testing (CT) and MBT. Finally, to achieve the last subgoal, we performed a comparative analysis between test suites developed using MBT, CT, and manual industrial practices. The results showed that the MBT-generated test suites using the edge coverage criterion tend to cover each requirement multiple times while achieving the same level of requirement coverage as the manually written test suites. Moreover, MBT provided higher Modified Decision and Condition Coverage (MC/DC) than CT and manual testing. On the other hand, CT came out as the most efficient technique in terms of the time required to generate and execute tests as well as achieving the highest fault detection rate with 3-ways and 4-ways interaction strength. Hence, based on the results, we conclude that manual industrial testing will benefit from MBT and CT for improved coverage and fault detection.