mdh.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors
Westermo Research and Development AB, Västerås, Sweden.
Mälardalens högskola, Akademin för innovation, design och teknik, Inbyggda system.ORCID-id: 0000-0002-5032-2310
Mälardalens högskola, Akademin för innovation, design och teknik, Inbyggda system.ORCID-id: 0000-0003-0611-2655
Mälardalens högskola, Akademin för innovation, design och teknik, Inbyggda system.
Visa övriga samt affilieringar
2016 (Engelska)Ingår i: 27th International Symposium on Software Reliability Engineering ISSRE'16, 2016Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We propose a new method of determining an effective ordering of regression test cases, and describe its implementation as an automated tool called SuiteBuilder developed by Westermo Research and Development AB. The tool generates an efficient order to run the cases in an existing test suite by using expected or observed test duration and combining priorities of multiple factors associated with test cases, including previous fault detection success, interval since last executed, and modifications to the code tested. The method and tool were developed to address problems in the traditional process of regression testing, such as lack of time to run a complete regression suite, failure to detect bugs in time, and tests that are repeatedly omitted. The tool has been integrated into the existing nightly test framework for Westermo software that runs on large-scale data communication systems.  In experimental evaluation of the tool, we found significant improvement in regression testing results. The re-ordered test suites finish within the available time, the majority of fault-detecting test cases are located in the first third of the suite, no important test case is omitted, and the necessity for manual work on the suites is greatly reduced.

Ort, förlag, år, upplaga, sidor
2016.
Nyckelord [en]
Automation; Data communication systems; Fault detection; Program debugging; Regression analysis; Software reliability Automated systems; Experience report; Experimental evaluation; Large scale data; Regression suites; Regression testing; Regression tests; Research and development
Nationell ämneskategori
Datorsystem
Identifikatorer
URN: urn:nbn:se:mdh:diva-33812DOI: 10.1109/ISSRE.2016.23ISI: 000391437700002Scopus ID: 2-s2.0-85013287871OAI: oai:DiVA.org:mdh-33812DiVA, id: diva2:1048574
Konferens
27th International Symposium on Software Reliability Engineering ISSRE'16, 23-27 Oct 2016, Ottawa, Canada
Tillgänglig från: 2016-11-21 Skapad: 2016-11-21 Senast uppdaterad: 2018-10-02Bibliografiskt granskad
Ingår i avhandling
1. Automated System Level Software Testing of Networked Embedded Systems
Öppna denna publikation i ny flik eller fönster >>Automated System Level Software Testing of Networked Embedded Systems
2018 (Engelska)Licentiatavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Embedded systems are ubiquitous and play critical roles in management systems for industry and transport. Software failures in these domains may lead to loss of production or even loss of life, so the software in these systems needs to be reliable. Software testing is a standard approach for quality assurance of embedded software, and many software development processes strive for test automation. However, important challenges for successful software test automation are: lack of time for testing, lack of test environment availability, and an excess of test results information that renders decision-making hard.

In this thesis these challenges are tackled in three ways. First, in order to combat lack of time for testing, a method for automated system level regression test selection was implemented and evaluated using data from several years of nightly testing in a real world industrial setting. It was shown that automated test selection can be integrated into system level nightly testing and solve problems such as nightly testing not finishing on time. Second, in order to improve the hardware coverage of devices in the test environment and how test cases map to hardware, an algorithm based on the subgraph isomorphism problem was implemented and evaluated using industrial data. This implementation was significantly faster than the previous implementation, and the mapping process was done in such a way that hardware coverage increased over iterations. Third, to better understand decision-making in critical steps of the software development process in an industrial setting, two empirical studies were conducted. The results showed how visualizations and a test results database support decision-making. Results also describe the overall flow of information in software testing: from developers to hardware, and back to developers via the test results database.

Automated system level software testing of networked embedded systems can be difficult to achieve. This thesis addresses several important challenges and provides results that are of interest both to industrial practitioners and researchers.

Ort, förlag, år, upplaga, sidor
Västerås: Mälardalen University, 2018
Serie
Mälardalen University Press Licentiate Theses, ISSN 1651-9256 ; 275
Nationell ämneskategori
Programvaruteknik
Forskningsämne
datavetenskap
Identifikatorer
urn:nbn:se:mdh:diva-41109 (URN)978-91-7485-408-4 (ISBN)
Presentation
2018-11-13, Room Zeta, Mälardalens högskola, Västerås, 13:15 (Engelska)
Opponent
Handledare
Tillgänglig från: 2018-10-02 Skapad: 2018-10-02 Senast uppdaterad: 2018-10-09Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltextScopus

Personposter BETA

Strandberg, Per Erik

Sök vidare i DiVA

Av författaren/redaktören
Strandberg, Per ErikSundmark, DanielAfzal, WasifOstrand, ThomasWeyuker, Elaine
Av organisationen
Inbyggda system
Datorsystem

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 119 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf