mdh.sePublikasjoner
Endre søk
Link to record
Permanent link

Direct link
BETA
Publikasjoner (9 av 9) Visa alla publikasjoner
Flemström, D., Enoiu, E. P., Afzal, W., Daniel, S., Gustafsson, T. & Kobetski, A. (2018). From natural language requirements to passive test cases using guarded assertions. In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018: . Paper presented at 18th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2018, 16 July 2018 through 20 July 2018 (pp. 470-481). Institute of Electrical and Electronics Engineers Inc.
Åpne denne publikasjonen i ny fane eller vindu >>From natural language requirements to passive test cases using guarded assertions
Vise andre…
2018 (engelsk)Inngår i: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, s. 470-481Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

In large-scale embedded system development, requirements are often expressed in natural language. Translating these requirements to executable test cases, while keeping the test cases and requirements aligned, is a challenging task. While such a transformation typically requires extensive domain knowledge, we show that a systematic process in combination with passive testing would facilitate the translation as well as linking the requirements to tests. Passive testing approaches observe the behavior of the system and test their correctness without interfering with the normal behavior. We use a specific approach to passive testing: guarded assertions (G/A). This paper presents a method for transforming system requirements expressed in natural language into G/As. We further present a proof of concept evaluation, performed at Bombardier Transportation Sweden AB, in which we show how the process would be used, together with practical advice of the reasoning behind the translation steps.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers Inc., 2018
Emneord
Computer software selection and evaluation, Embedded systems, Natural language processing systems, Software reliability, Bombardier Transportation, Domain knowledge, Large scale embedded systems, Natural language requirements, Natural languages, Proof of concept, System requirements, Systematic process, Translation (languages)
HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-40744 (URN)10.1109/QRS.2018.00060 (DOI)2-s2.0-85052319900 (Scopus ID)9781538677575 (ISBN)
Konferanse
18th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2018, 16 July 2018 through 20 July 2018
Tilgjengelig fra: 2018-09-07 Laget: 2018-09-07 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Flemström, D., Pasqualina, P., Daniel, S., Afzal, W. & Bohlin, M. (2018). Similarity-Based Prioritization of Test Case Automation. Software quality journal, 26(4), 1421-1449
Åpne denne publikasjonen i ny fane eller vindu >>Similarity-Based Prioritization of Test Case Automation
Vise andre…
2018 (engelsk)Inngår i: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 26, nr 4, s. 1421-1449Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.

HSV kategori
Forskningsprogram
datavetenskap
Identifikatorer
urn:nbn:se:mdh:diva-35055 (URN)10.1007/s11219-017-9401-7 (DOI)000448856400008 ()2-s2.0-85043389019 (Scopus ID)
Prosjekter
IMPRINT
Forskningsfinansiär
Vinnova, 2014-03397Knowledge Foundation, 20130085Knowledge Foundation, 20160139
Tilgjengelig fra: 2017-03-22 Laget: 2017-03-22 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Flemström, D. (2017). Similarity-Based Test Effort Reduction. (Licentiate dissertation). Västerås: Mälardalen University Press
Åpne denne publikasjonen i ny fane eller vindu >>Similarity-Based Test Effort Reduction
2017 (engelsk)Licentiatavhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Embedded computer systems are all around us. We find them in everything, from dishwashers to cars and airplanes. They must always work correctly and moreover, often within certain time constraints. The software of such a system can be very large and complex, e.g. in the case of a car or a train. Hence, we develop the software for embedded systems in smaller, manageable, parts. These parts can be successively integrated, until they form the complete software for the embedded system, possibly at different levels. This phase of the development process is called the system integration phase and is one of the most critical phases in the development of embedded systems. In this phase, substantial effort is spent on testing activities.

Studies have found that a considerable amount of test effort is wasteful due to people, unknowingly or by necessity, performing similar (or even overlapping) test activities. Consequently, test cases may end up to be similar, partially or wholly. We identified such test similarities in a case study of 2500 test cases, written in natural language, from four different projects in the embedded vehicular domain. Such information can be used for reducing effort when maintaining or automating similar test cases.

In another case study in the same domain, we investigated several approaches for prioritizing test cases to automate with the objective to reduce manual test effort as quick as possible given that similar automated tests could be reused (similarity-based reuse). We analyzed how the automation order affects the test effort for four projects with a total of 3919 integration test cases, written in natural language. The results showed that similarity-based reuse of automated test case script code, and the best-performing automation order can reduce the expected manual test effort with 20 percentage points.

Another way of reducing test effort is to reuse test artifacts from one level of integration to another, instead of duplicating them. We studied such reuse methods, that we denote vertical reuse, in a systematic mapping study. While the results from of our systematic mapping study showed the viability of vertical test reuse methods, our industrial case studies showed that keeping track of similarities and test overlaps is both possible and feasible for test effort reduction. We further conclude that the test case automation order affects the manual test execution effort when there exist similar steps that cannot be removed, but are possible to reuse with respect to test script code.

Abstract [sv]

Inbyggda datorsystem finns överallt omkring oss idag; i allt från diskmaskiner till bilar och flygplan. Ofta finns det stränga krav på såväl korrekt funktion som svarstider. Eftersom programvaran i det inbyggda systemet i till exempel en bil är väldigt stor och komplex, utvecklar man programvaran i mindre delar som sedan successivt integreras till det färdiga datorsystemet. Denna del av utvecklingsprocessen kallas systemintegrationsfasen och är en av de största och mest kritiska faserna när man utvecklar inbyggda system. Integrationen kan utföras i ett antal nivåer med utförliga tester av mjukvaran på varje nivå. Detta innebär att det krävs ett avsevärt arbete för att testa mjukvaran. Om man kan minska detta arbete, givetvis utan att ge avkall på testningens kvalitet, förväntas det få en stor effekt på det totala utvecklingsarbetet.

Studier har visat att en icke försumbar del av testarbetet är slöseri på grund av att människor, omedvetet eller av nödvändighet, utför likartade (eller till och med överlappande) testaktiviteter. En konsekvens är att testfall riskerar att bli helt eller delvis lika. Vi identifierade sådana likheter i en fallstudie med 2500 manuella testfall, skrivna på vanligt språk, från fyra projekt inom fordonsindustrin. Information om likheter och överlapp kan användas för att, till exempel, minska arbetsåtgången vid underhåll eller när man översätter testfall till kod så att de kan utföras automatiskt av en dator (automatisering). I en annan studie inom samma domän, undersökte vi flera metoder för att prioritera arbetsordningen vid automatisering av testfall där liknande testfall kunde återanvändas. I studien analyserade vi hur denna ordning påverkar arbetsmängden för fyra industriprojekt med totalt 3919 integrationstestfall skrivna på vanligt språk. Resultaten visar att den bästa ordningen kan minska testarbetet avsevärt. En förutsättning för detta är att redan översatta delar av testfall kan återanvändas för att slippa översätta liknande testfall igen.

En annan väg för att minska testarbetet är att återanvända testfall och information mellan integrationsnivåer. Vi har kartlagt metoder och motiv kring sådan återanvändning i en systematisk mappningsstudie. Våra fallstudier visar vidare att det både är genomförbart och lönsamt att hålla reda på likheter i testfallen. Slutligen konstaterar vi att arbetsinsatsen för manuell testning påverkas av automationsordningen när det är möjligt att återanvända redan översatta delar av liknande testfall.

sted, utgiver, år, opplag, sider
Västerås: Mälardalen University Press, 2017
Serie
Mälardalen University Press Licentiate Theses, ISSN 1651-9256 ; 257
Emneord
Software Testing
HSV kategori
Forskningsprogram
datavetenskap
Identifikatorer
urn:nbn:se:mdh:diva-35053 (URN)978-91-7485-318-6 (ISBN)
Presentation
2017-04-21, Pi, Mälardalen University, Västerås, 13:15 (engelsk)
Opponent
Veileder
Prosjekter
IMPRINT
Forskningsfinansiär
VINNOVA, 2014-03397
Tilgjengelig fra: 2017-03-23 Laget: 2017-03-22 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Flemström, D. (2017). Tech. Report: Similarity Function Evaluation. Västerås, Sweden: Mälardalen Real-Time Research Centre, Mälardalen University
Åpne denne publikasjonen i ny fane eller vindu >>Tech. Report: Similarity Function Evaluation
2017 (engelsk)Rapport (Annet vitenskapelig)
Abstract [en]

This report presents details on and an in-depth evaluation of a similarity function used for detecting similar test steps in manual test cases, written in natural language. Using an industrial data set of 65 000 test steps, we show that even though the similarity function builds on standard functions from the open source data base Postgres, it is capable of finding similarities in parity of what the state of the art suggests. Rather few miss classifications were found. We also show that by fine tuning the function, the number of clusters of similar can be reduced by 13%. Manual inspection further shows that there is potential to reduce the set of clusters even more.

sted, utgiver, år, opplag, sider
Västerås, Sweden: Mälardalen Real-Time Research Centre, Mälardalen University, 2017
Serie
MRTC Reports, ISSN 1404-3041
Emneord
Software Engineering, Sofware Testing, Similarity Detection
HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-35489 (URN)MDH-MRTC-315/2017-1-SE (ISRN)
Tilgjengelig fra: 2017-06-01 Laget: 2017-06-01 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Flemström, D., Afzal, W. & Sundmark, D. (2016). Exploring Test Overlap in System Integration: An Industrial Case Study. In: 42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016: . Paper presented at 42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016, 31 Aug 2016, Cyprus, Cyprus (pp. 303-312).
Åpne denne publikasjonen i ny fane eller vindu >>Exploring Test Overlap in System Integration: An Industrial Case Study
2016 (engelsk)Inngår i: 42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016, 2016, s. 303-312Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Tougher safety regulations, global competition and ever increasing complexity of embedded software puts extensive pressure on the effectiveness of the software testing procedures. Previous studies have found that there exist overlaps (i.e., multiple instances of highly similar test cases) and even redundancies in the software testing process. Such overlap has been found between versions, variants and integration levels, but primarily at unit test level. Given large embedded systems involving many subsystems, does overlap exist within the system integration testing as well? In this paper, we present an industrial case study, aiming to a) evaluate if there exist test overlaps within the given context, b) if so, investigate how these overlaps are distributed, and c) find ways of reducing test effort by investigating how the knowledge of overlaps and their distribution may be used for finding candidate test cases for automation, maintenance or even removal. We have studied manual test cases, written in natural language, at a large vehicular manufacturer in Sweden. In particular, we have collected and analyzed test cases from the system integration testing levels of four different projects of a vehicle control management system. Using a similarity function, we evaluate if any overlaps between test cases exist, and where. We found that overlaps do exist within the system integration level, particularly in the form of partial test step sequences. However, very few test cases overlapped in their entirety. Some candidates for test step automation and update propagation were identified, but none for easy removal.

HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-32865 (URN)10.1109/SEAA.2016.34 (DOI)000386649000044 ()2-s2.0-85020747520 (Scopus ID)978-1-5090-2819-1 (ISBN)
Konferanse
42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016, 31 Aug 2016, Cyprus, Cyprus
Prosjekter
IMPRINT - Innovative Model-Based Product Integration Testing (Vinnova)
Tilgjengelig fra: 2016-09-23 Laget: 2016-08-24 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Flemström, D., Sundmark, D. & Afzal, W. (2015). Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study. In: The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15: . Paper presented at The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, 26-28 Aug 2015, Funchal, Madeira, Portugal (pp. 317-324).
Åpne denne publikasjonen i ny fane eller vindu >>Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study
2015 (engelsk)Inngår i: The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, 2015, s. 317-324Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Vertical test reuse refers to the the reuse of test cases or other test artifacts over different integration levels in the software or system engineering process. Vertical test reuse has previously been proposed for reducing test effort and improving test effectiveness, particularly for embedded system development. The goal of this study is to provide an overview of the state of the art in the field of vertical test reuse for embedded system development. For this purpose, a systematic mapping study has been performed, identifying 11 papers on vertical test reuse for embedded systems. The primary result from the mapping is a classification of published work on vertical test reuse in the embedded system domain, covering motivations for reuse, reuse techniques, test levels and reusable test artifacts considered, and to what extent the effects of reuse have been evaluated.

HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-28164 (URN)10.1109/SEAA.2015.46 (DOI)000380478300046 ()2-s2.0-84958235305 (Scopus ID)978-1-4673-7585-6 (ISBN)
Eksternt samarbeid:
Konferanse
The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, 26-28 Aug 2015, Funchal, Madeira, Portugal
Prosjekter
IMPRINT - Innovative Model-Based Product Integration Testing (Vinnova)
Tilgjengelig fra: 2015-06-12 Laget: 2015-06-08 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Larsson, T. & Flemström, D. (2007). Improving Introductory Programming Courses by Using a Simple Accelerated Graphics Library. In: SIGRAD 2007: Conference Proceedings. Paper presented at SIGRAD 2007. The Annual SIGRAD Conference; Special Theme: Computer Graphics in Healthcare; November 28–29; 2007; Uppsala; Sweden (pp. 24-30).
Åpne denne publikasjonen i ny fane eller vindu >>Improving Introductory Programming Courses by Using a Simple Accelerated Graphics Library
2007 (engelsk)Inngår i: SIGRAD 2007: Conference Proceedings, 2007, s. 24-30Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

We present a platform independent and hardware accelerated graphics library, which has been found to be a suitable educational tool for novice programmers. The purpose of the library is to change the nature of the assignments and projects used in introductory programming courses to something that fascinate and stimulate the students, e.g., game creation. We describe our experiences from using the presented graphics library in four different course instances of our introductory C++ course. The course surveys show that most students found the approach interesting and fun. As faculty, we could clearly see how many students became highly engaged in their projects and some of them accomplished solutions way beyond our expectations. In particular, compared to the programming courses we have given in the past, in which a standard framework for creating window applications was used, we have noticed a significant improvement in terms of the quality of the students' project solutions.

Serie
Linköping Electronic Conference Proceedings, ISSN 1650-3686
HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-7087 (URN)978-91-7393-990-4 (ISBN)
Konferanse
SIGRAD 2007. The Annual SIGRAD Conference; Special Theme: Computer Graphics in Healthcare; November 28–29; 2007; Uppsala; Sweden
Tilgjengelig fra: 2009-09-25 Laget: 2009-09-25 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Lüders, F., Flemström, D., Wall, A. & Crnkovic, I. (2006). A Prototype Tool for Software Component Services in Embedded Real-Time Systems. In: Component-Based Software Engineering: 9th International Symposium, CBSE 2006, Västerås, Sweden, June 29 - July 1, 2006. Proceedings (pp. 222-237). Springer
Åpne denne publikasjonen i ny fane eller vindu >>A Prototype Tool for Software Component Services in Embedded Real-Time Systems
2006 (engelsk)Inngår i: Component-Based Software Engineering: 9th International Symposium, CBSE 2006, Västerås, Sweden, June 29 - July 1, 2006. Proceedings, Springer, 2006, s. 222-237Kapittel i bok, del av antologi (Fagfellevurdert)
Abstract [en]

The use of software component models has become popular during the last decade, in particular in the development of software for desktop applications and distributed information systems. However, such models have not been widely used in the domain of embedded real-time systems. There is a considerable amount of research on component models for embedded real-time systems, or even narrower application domains, which focuses on source code components and statically configured systems. This paper explores an alternative approach by laying the groundwork for a component model based on binary components and targeting the broader domain of embedded real-time systems. The work is inspired by component models for the desktop and information systems domains in the sense that a basic component model is extended with a set of services for the targeted application domain. A prototype tool for supporting these services is presented and its use illustrated by a control application.

sted, utgiver, år, opplag, sider
Springer, 2006
Serie
Lecture Notes in Computer Science, ISSN 0302-9743 ; 4063
HSV kategori
Identifikatorer
urn:nbn:se:mdh:diva-4034 (URN)10.1007/11783565_16 (DOI)000239520500016 ()2-s2.0-33746401971 (Scopus ID)3-540-35628-2 (ISBN)
Tilgjengelig fra: 2006-11-20 Laget: 2006-11-20 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Lüders, F., Flemström, D. & Wall, A. (2006). Software components services for embedded real-time systems. In: Proceedings - 5th Working IEEE/IFIP Conference on Software Architecture, WICSA 2005: . Paper presented at 5th Working IEEE/IFIP Conference on Software Architecture, WICSA 2005, 6 November 2005 through 10 November 2005, Pittsburgh, PA (pp. 278-279).
Åpne denne publikasjonen i ny fane eller vindu >>Software components services for embedded real-time systems
2006 (engelsk)Inngår i: Proceedings - 5th Working IEEE/IFIP Conference on Software Architecture, WICSA 2005, 2006, s. 278-279Konferansepaper, Publicerat paper (Fagfellevurdert)
Identifikatorer
urn:nbn:se:mdh:diva-31912 (URN)10.1109/WICSA.2005.66 (DOI)000237618100058 ()2-s2.0-33947128270 (Scopus ID)0769525482 (ISBN)9780769525488 (ISBN)
Konferanse
5th Working IEEE/IFIP Conference on Software Architecture, WICSA 2005, 6 November 2005 through 10 November 2005, Pittsburgh, PA
Tilgjengelig fra: 2016-06-15 Laget: 2016-06-14 Sist oppdatert: 2019-12-14bibliografisk kontrollert
Organisasjoner
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0001-8096-3592