https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Lessons learned from replicating a study on information-retrieval-based test case prioritization
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Blekinge Inst Technol, Karlskrona, Sweden.ORCID iD: 0000-0001-8177-4355
Ericsson Sweden AB, Karlskrona, Sweden .
Univ Appl Sci Flensburg, Flensburg, Germany .
Blekinge Inst Technol, Karlskrona, Sweden .
(English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367Article in journal (Refereed) Epub ahead of print
Abstract [en]

Replication studies help solidify and extend knowledge by evaluating previous studies' findings. Software engineering literature showed that too few replications are conducted focusing on software artifacts without the involvement of humans. This study aims to replicate an artifact-based study on software testing to address the gap related to replications. In this investigation, we focus on (i) providing a step-by-step guide of the replication, reflecting on challenges when replicating artifact-based testing research and (ii) evaluating the replicated study concerning the validity and robustness of the findings. We replicate a test case prioritization technique proposed by Kwon et al. We replicated the original study using six software programs, four from the original study and two additional software programs. We automated the steps of the original study using a Jupyter notebook to support future replications. Various general factors facilitating replications are identified, such as (1) the importance of documentation; (2) the need for assistance from the original authors; (3) issues in the maintenance of open-source repositories (e.g., concerning needed software dependencies, versioning); and (4) availability of scripts. We also noted observations specific to the study and its context, such as insights from using different mutation tools and strategies for mutant generation. We conclude that the study by Kwon et al. is partially replicable for small software programs and could be automated to facilitate software practitioners, given the availability of required information. However, it is hard to implement the technique for large software programs with the current guidelines. Based on lessons learned, we suggest that the authors of original studies need to publish their data and experimental setup to support the external replications.

National Category
Software Engineering
Identifiers
URN: urn:nbn:se:mdh:diva-64642DOI: 10.1007/s11219-023-09650-4ISI: 001084224100001Scopus ID: 2-s2.0-85174265778OAI: oai:DiVA.org:mdh-64642DiVA, id: diva2:1808950
Available from: 2023-11-01 Created: 2023-11-01 Last updated: 2024-01-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Minhas, Nasir Mehmood
By organisation
Embedded Systems
In the same journal
Software quality journal
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 42 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf