https://www.mdu.se/

mdu.sePublications
Planned maintenance
A system upgrade is planned for 10/12-2024, at 12:00-13:00. During this time DiVA will be unavailable.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
An Empirical Evaluation of System-Level Test Effectiveness for Safety-Critical Software
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0001-8746-7209
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0003-0611-2655
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0003-2416-4205
2023 (English)In: International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings, Science and Technology Publications, Lda , 2023, p. 293-305Conference paper, Published paper (Refereed)
Abstract [en]

Combinatorial Testing (CT) and Model-Based Testing (MBT) are two recognized test generation techniques. The evidence of their fault detection effectiveness and comparison with industrial state-of-the-practice is still scarce, more so at the system level for safety-critical systems, such as those found in trains. We use mutation analysis to perform a comparative evaluation of CT, MBT, and industrial manual testing in terms of their fault detection effectiveness using an industrial case study of the safety-critical train control management system. We examine the fault detection rate per mutant and relationship between the mutation scores and structural coverage using Modified Condition Decision Coverage (MC/DC). Our results show that CT 3-ways, CT 4-ways, and MBT provide higher mutation scores. MBT did not perform better in detecting 'Logic Replacement Operator-Improved' mutants when compared with the other techniques, while manual testing struggled to find 'Logic Block Replacement Operator' mutants. None of the test suites were able to find 'Time Block Replacement Operator' mutants. CT 2-ways was found to be the least effective test technique. MBT-generated test suite achieved the highest MC/DC coverage. We also found a generally consistent positive relationship between MC/DC coverage and mutation scores for all test suites.

Place, publisher, year, edition, pages
Science and Technology Publications, Lda , 2023. p. 293-305
Keywords [en]
Fault Detection Effectiveness, Safety-Critical Software, System-Level Tests
National Category
Computer Systems
Identifiers
URN: urn:nbn:se:mdh:diva-62926DOI: 10.5220/0011756800003464ISI: 001119034200025Scopus ID: 2-s2.0-85160537558ISBN: 9789897586477 (print)OAI: oai:DiVA.org:mdh-62926DiVA, id: diva2:1763666
Conference
18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023, Prague, Czech Republic, 24/4-25/4, 2023
Available from: 2023-06-07 Created: 2023-06-07 Last updated: 2024-01-17Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Zafar, Muhammad NoumanAfzal, WasifEnoiu, Eduard Paul

Search in DiVA

By author/editor
Zafar, Muhammad NoumanAfzal, WasifEnoiu, Eduard Paul
By organisation
Embedded Systems
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 44 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf