https://www.mdu.se/

Model-based testing (MBT) generates test cases through a model representing the software under test (SUT). The generated abstract test cases need to be transformed into concrete or executable test scripts. Despite the benefits offered by MBT, its industrial adoption is slow. This paper aims to propose a Model-Based Test scrIpt GenEration fRamework (TIGER) based on GraphWalker (GW), an open-source MBT tool, to evaluate the accuracy of generated test scripts to reflect real-world scenarios defined by the model, and to report on the findings of an industrial survey on MBT adoption. We have validated the robustness of the TIGER using an industrial case study from Alstom Rail AB, Sweden. We have injected faults into the model of the SUT based on three mutation operators to generate faulty test scripts. The aim of generating faulty test scripts is to produce failing test steps and to guarantee the absence of faults in the SUT. Moreover, we have also generated the test scripts using the correct version of the model and executed them to analyze the behavior of the generated test scripts in comparison with manually written test scripts. The experimental results show that the generated test scripts are executable, provide 100% requirements coverage, and can be used to uncover faults at the software-in-the-loop simulation level of system testing. Additionally, the analysis of the survey data reveals that MBT can address most of the identified testing challenges, but there remain certain barriers to its adoption.

Several studies have shown Model-based Testing (MBT) as an efficient technique for generating fault-effective test cases. However, the automatic generation of test cases is compromised with redundant test cases providing no additional value to the coverage or fault detection effectiveness while impacting test execution efficiency, especially, in a dynamic development environment where providing timely feedback is crucial. These redundant test cases need to be discarded to minimize the test suite size and their effect on the execution cost and efficiency of a test suite. Reducing a test suite becomes challenging for black box testing at the system level when no information regarding the coverage and fault detection effectiveness of the test suite exists. Hence, in this paper, we have presented a test suite optimization approach leveraging different machine learning algorithms, a greedy algorithm, and a similarity measure. The proposed approach generates a reduced test suite by identifying and eliminating redundant test cases from an MBT-generated test suite while having minimal impact on the fault detection rate. We have also performed a comparative evaluation of the optimized test suites with the MBT-generated and manually created test suites in terms of fault detection effectiveness and test execution efficiency using an industrial case study from Alstom Rail AB, Sweden. The results show a significant reduction of 85% to 92% in the size of the test suite. Moreover, we also found the test execution time of the optimized test suite equivalent to the manually created tests and a fault detection rate within the range of 95% to 100% for all test suites under observation.

Combinatorial Testing (CT) and Model-Based Testing (MBT) are two recognized test generation techniques. The evidence of their fault detection effectiveness and comparison with industrial state-of-the-practice is still scarce, more so at the system level for safety-critical systems, such as those found in trains. We use mutation analysis to perform a comparative evaluation of CT, MBT, and industrial manual testing in terms of their fault detection effectiveness using an industrial case study of the safety-critical train control management system. We examine the fault detection rate per mutant and relationship between the mutation scores and structural coverage using Modified Condition Decision Coverage (MC/DC). Our results show that CT 3-ways, CT 4-ways, and MBT provide higher mutation scores. MBT did not perform better in detecting 'Logic Replacement Operator-Improved' mutants when compared with the other techniques, while manual testing struggled to find 'Logic Block Replacement Operator' mutants. None of the test suites were able to find 'Time Block Replacement Operator' mutants. CT 2-ways was found to be the least effective test technique. MBT-generated test suite achieved the highest MC/DC coverage. We also found a generally consistent positive relationship between MC/DC coverage and mutation scores for all test suites.

Adequate testing of safety-critical systems is vital to ensure correct functional and non-functional operations. Previous research has shown that testing such systems requires a lot of effort, thus automated testing techniques have found a certain degree of success. However, automated testing has not replaced the need for manual testing, rather a common industrial practice exhibits a balance between automated and manual testing. In this respect, comparing manual testing with automated testing techniques continues to be an interesting topic to investigate. The need for this investigation is most apparent at system-level testing of industrial systems, where there is a lack of results on how different testing techniques perform concerning both structural and system-level metrics such as Modified Condition/Decision Coverage (MC/DC) and requirement coverage. In addition to the coverage, the cost of these techniques will also determine their efficiency and thus practical viability. In this paper, we have developed cost models for efficiency measurement and performed an experimental evaluation of manual testing, model-based testing (MBT), and combinatorial testing (CT) in terms of MC/DC and requirement coverage. The evaluation is done in an industrial context of a safety-critical system that controls several functions on-board the passenger trains. We have reported the dominant conditions of MC/DC affected by each technique while generating MC/DC adequate test suites. Moreover, we investigated differences and overlaps of test cases generated by each of the three techniques. The results showed that all test suites achieved 100% requirement coverage except the test suite generated by the pairwise testing strategy. However, MBT-generated test suites were more MC/DC adequate and provided a higher number of both similar and unique test cases. Moreover, unique test cases generated by MBT had an observable effect on MC/DC, which will complement manual testing to increase MC/DC coverage. The least dominant MC/DC condition fulfilled by the generated test cases by all three techniques is the 'independent effect of a condition on the outcomes of a decision'. Lastly, the evaluation also showed CT as the most efficient testing technique amongst the three in terms of time required for test generation and execution, but with an added cost parameter of manual identification of expected outcomes. CCS CONCEPTS •Computer systems organization →Embedded software; • Software and its engineering →Software verification and validation.

System-level testing of safety-critical embedded systems is complex and costly. MBT has shown promising results in terms of fault detection effectiveness and efficiency of test generation and execution. However, the industrial adoption of MBT approaches is slow and limited to specific industries and domains. Moreover, the strengths and weaknesses of MBT in industrial settings need to be thoroughly evaluated to find an optimal testing strategy.

The objective of this thesis is to adapt, compare, and evaluate the effectiveness and efficiency of MBT to help industrial practitioners in the testing of safety-critical embedded software. We have divided this objective into three subgoals. To achieve the first subgoal, we have explored multiple state-of-the- art MBT tools and evaluated the selected tool, GraphWalker (GW), in terms of modeling notations, generation algorithm, stopping conditions, and model completeness. To achieve the second subgoal, we have proposed a Model-Based Test scrIpt GenEration fRamework (TIGER), based on GW, to generate system-level test artifacts (i.e., test cases and test scripts). Based on the proposed framework, we implemented two test script generation tools for combinatorial testing (CT) and MBT. Finally, to achieve the last subgoal, we performed a comparative analysis between test suites developed using MBT, CT, and manual industrial practices. 

The results showed that the MBT-generated test suites using the edge coverage criterion tend to cover each requirement multiple times while achieving the same level of requirement coverage as the manually written test suites. Moreover, MBT provided higher Modified Decision and Condition Coverage (MC/DC) than CT and manual testing. On the other hand, CT came out as the most efficient technique in terms of the time required to generate and execute tests as well as achieving the highest fault detection rate with 3-ways and 4-ways interaction strength. Hence, based on the results, we conclude that manual industrial testing will benefit from MBT and CT for improved coverage and fault detection.

Testning på systemnivå av säkerhetskritiska inbyggda system är komplext och kostsamt. MBT har visat lovande resultat när det gäller feldetekteringseffektivitet och effektivitet i testgenerering och testutförande. Den industriella utvecklingen av MBT-metoder är dock långsam och begränsad till specifika branscher och domäner. Dessutom måste styrkorna och svagheterna hos MBT i industriella miljöer utvärderas noggrant för att hitta en optimal teststrategi.

Syftet med denna avhandling är att anpassa, jämföra och utvärdera effektiviteten hos MBT för att hjälpa industriutövare att testa säkerhetskritisk inbyggd programvara. Vi har delat in detta mål i tre delmål. För att uppnå det första delmålet har vi utforskat flera av de främsta MBT-verktygen och utvärderat det valda verktyget, GraphWalker (GW), i termer av modelleringsnotationer, genereringsalgoritm, stoppförhållanden och modellfullständighet. För att uppnå det andra delmålet har vi föreslagit ett Model-Based Test ScrIpt Generation Framework (TIGER), baserat på GW, för att generera testartefakter på systemnivå (d.v.s. testfall och testskript). Baserat på det föreslagna ramverket implementerade vi två testskriptgenereringsverktyg för kombinatorisk testning (CT) och MBT. Slutligen, för att uppnå det sista delmålet, utförde vi en jämförande analys mellan testsviter utvecklade med MBT, CT och manuell industriell praxis.

Resultaten visade att de MBT-genererade testsviterna som använder kanttäckningskriteriet tenderar att täcka varje krav flera gånger samtidigt som de uppnår samma nivå av kravtäckning som de manuellt skrivna testsviterna. Dessutom gav MBT högre Modified Decision and Condition Coverage (MC/DC) än CT och manuell testning. Å andra sidan kom CT ut som den mest effektiva tekniken när det gäller den tid som krävs för att generera och utföra tester samt att uppnå den högsta feldetekteringshastigheten med 3-vägs och 4-vägs interaktionsstyrka. Baserat på resultaten drar vi därför slutsatsen att manuell industriell testning kommer att dra nytta av MBT och CT för förbättrad täckning och feldetektering.

The abstract test cases generated through modelbased testing (MBT) need to be concretized to make them executable on the software under test (SUT). Multiple researchers proposed different solutions, e.g., by utilizing adapters for concretization of abstract test cases and generation of testscripts. In this paper, we propose our Model-Based Test scrIptGenEration fRamework (TIGER) based on GraphWalker, an open source MBT tool. The framework is capable of generating test scripts for embedded software controlling functions of a cyber physical system such as passenger trains developed at Bombardier Transportation AB. The framework follows somedefined mapping rules for the concretization of abstract test cases.We have evaluated the generated test scripts using an industrial case study in terms of fault detection. We have induced faultsin the model of the SUT based on three mutation operatorsto generate faulty test scripts. The aim of generating faulty test scripts is to produce failed test steps and to guarantee the absence of faults in the SUT. Moreover, we have also generated the test scripts using the correct version of the model and executed it to analyse the behaviour of the generated test scripts in comparisonwith manually-written test scripts. The results show that the test scripts generated by GW using the proposed frameworkare executable, provide 100% requirements coverage and can beused to uncover faults at software-in-the-loop simulation level of sub-system testing.

In recent years, smartphones have improved significantly because of their rich set of features, services and their memory capacity and processing speed. Consequently, complex applications (e.g., apps used for home automation, high performance games etc.) are being developed for users to provide on hand access to different functions of such apps. However, these apps are often energy-inefficient and can quickly drain the battery power of smartphones due to improper usage of resources. Such kind of energy inefficiencies in smartphone apps can be categorized into energy bugs and energy hotspots. When an app consumes more power due to some fault or failure even after closing or termination and prevents the smartphone from becoming idle, an energy bug occurs. Whereas, energy hotspot is caused due to abnormal usage of energy by an app during its execution. Execution of an app with an energy bug or energy hotspot can drastically shorten the lifetime of battery. Moreover, it also causes serious energy drain issues in smartphones due to which user experience is severely impacted. So, to cater the problem of energy inefficiencies, we have to identify the energy hotspots and bugs caused by such apps. In this paper, we report the findings of our systematic literature review to categorize and to structure the research findings that have been published in the area of energy consumption due to energy bugs in Android apps. Twenty-seven experimental studies are plotted. Besides, we have identified several challenges that need to be addressed in future. After going through the different studies, we found that existing approaches related to energy bugs either generate false positives or false negatives in addition to finding real energy bugs. We conclude that there is a need to utilize resources in a better way by making smartphone apps more energy-efficient and by implementing approaches that can automatically detect all possible energy bugs in Android apps.

Model-based testing (MBT) is a test design technique that supports the automation of software testing processes and generates test artefacts based on a system model representing behavioural aspects of the system under test (SUT). Previous research has shown some positive aspects of MBT such as low-cost test case generation and fault detection effectiveness. However, it is still a challenge for both practitioners and researchers to evaluate MBT tools and techniques in real, industrial settings. Consequently, the empirical evidence regarding the mainstream use, including the modelling and test case generation using MBT tools, is limited. In this paper, we report the results of a case study on applying GraphWalker, an open-source tool for MBT, on an industrial cyber-physical system (i.e., a Train Control Management System developed by Bombardier Transportation in Sweden), from modelling of real-world requirements and test specifications to test case generation. We evaluate the models of the SUT for completeness and representativeness, compare MBT with manual test cases written by practitioners using multiple attributes as well as share our experiences of selecting and using GraphWalker for industrial application. The results show that a model of the SUT created using both requirements and test specifications provides better understanding of the SUT from testers’ perspective, making it more complete and representative than the model created based only on the requirements specification alone. The generated model-based test cases are longer in terms of the number of test steps, achieve better edge coverage and can cover requirements more frequently in different orders while achieving the same level of requirements coverage as manually created test cases.

Model-based testing (MBT) has been previously used to validate embedded systems. However, (i) creation of a model conforming to the behavioural aspects of an embedded system, (ii) generation of executable test scripts and (iii) assessment of test verdict, re-quires a systematic process. In this paper, we have presented a three-phase tool-supported MBT workflow for the testing of an embedded system, that spans from requirements specification to test verdict assessment. The workflow starts with a simplistic, yet practical, application of a Domain-Specific Language (DSL) based on Gherkin-like style, which allows the requirements engineer to specify requirements and to extract information about model elements(i.e. states and transitions). This is done to assist the graphical modelling of the complete system under test (SUT). Later stages of the workflow generates an executable test script that runs on a domain-specific simulation platform. We have evaluated this tool-supported workflow by specifying the requirements, extracting information from the DSL and developing a model of a subsystem of the train control management system developed at Alstom Transport AB in Sweden. The C# test script generated from the SUT model is successfully executed at the Software-in-the-Loop (SIL) execution platform and test verdicts are visualized as a sequence of passed and failed test steps.

In this study, a three-dimensional numerical analysis has been done to investigate the heat transfer, pressure drop in the shell side and vortex shedding, tubes deformation due to fluid induced vibrations in the shell and tube heat exchanger (STHX). Threedimensional CFD and two-way FSI has performed with the commercial software ANSYS. To examine the thermo-hydraulic performance and induced vibrations in shell-and-tube heat exchangers with segmental/ helical/ clamping anti-vibration baffles and cylindrical/twisted tubes, numerical simulations are carried out. The numerical models show the thermo-hydraulic performances for the heat exchangers with segmental, helical and novel clamping anti-vibration baffles with cylindrical and square twisted tubes. The result shows that the use of square twisted tubes result in higher heat transfer rate as compared to cylindrical tubes. As far as pressure drop is concerned, it is also greater in the shell and tube heat exchangers with square twisted tubes for segmental, helical and anti-vibration baffles. The deformation in the tubes, velocity of the tubes and vortex shedding formation is minimum in STHX with clamping anti-vibration baffles than in STHXs with helical and segmental baffles.