https://www.mdu.se/

mdu.sePublications
Change search
Refine search result
12 1 - 50 of 90
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Abbas, Muhammad
    et al.
    RISE Res Inst Sweden, Västerås, Sweden..
    Ferrari, Alessio
    CNR, ISTI, Pisa, Italy..
    Shatnawi, Anas
    Berger Levrault, Montpellier, France..
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, Mehrdad
    RISE Res Inst Sweden, Västerås, Sweden..
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Correction to: On the relationship between similar requirements and similar software A case study in the railway domain (Jan, 10.1007/s00766-021-00370-4, 2022)2022In: Requirements Engineering, ISSN 0947-3602, E-ISSN 1432-010X, Vol. 27, no 3, p. 399-399Article in journal (Refereed)
  • 2.
    Abbas, Muhammad
    et al.
    RISE Res Inst Sweden, Västerås, Sweden.
    Ferrari, Alessio
    CNR ISTI, Pisa, Italy.
    Shatnawi, Anas
    Berger Levrault, Montpellier, France.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, Mehrdad
    RISE Res Inst Sweden, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    On the relationship between similar requirements and similar software: A case study in the railway domainIn: Requirements Engineering, ISSN 0947-3602, E-ISSN 1432-010XArticle in journal (Refereed)
    Abstract [en]

    Recommender systems for requirements are typically built on the assumption that similar requirements can be used as proxies to retrieve similar software. When a stakeholder proposes a new requirement, natural language processing (NLP)-based similarity metrics can be exploited to retrieve existing requirements, and in turn, identify previously developed code. Several NLP approaches for similarity computation between requirements are available. However, there is little empirical evidence on their effectiveness for code retrieval. This study compares different NLP approaches, from lexical ones to semantic, deep-learning techniques, and correlates the similarity among requirements with the similarity of their associated software. The evaluation is conducted on real-world requirements from two industrial projects from a railway company. Specifically, the most similar pairs of requirements across two industrial projects are automatically identified using six language models. Then, the trace links between requirements and software are used to identify the software pairs associated with each requirements pair. The software similarity between pairs is then automatically computed with JPLag. Finally, the correlation between requirements similarity and software similarity is evaluated to see which language model shows the highest correlation and is thus more appropriate for code retrieval. In addition, we perform a focus group with members of the company to collect qualitative data. Results show a moderately positive correlation between requirements similarity and software similarity, with the pre-trained deep learning-based BERT language model with preprocessing outperforming the other models. Practitioners confirm that requirements similarity is generally regarded as a proxy for software similarity. However, they also highlight that additional aspect comes into play when deciding software reuse, e.g., domain/project knowledge, information coming from test cases, and trace links. Our work is among the first ones to explore the relationship between requirements and software similarity from a quantitative and qualitative standpoint. This can be useful not only in recommender systems but also in other requirements engineering tasks in which similarity computation is relevant, such as tracing and change impact analysis.

  • 3.
    Abbas, Muhammad
    et al.
    Research Institutes of Sweden Västerås, Sweden.
    Inayat, Irum
    National University of Computer & Emerging Sciences Islamabad, Pakistan.
    Jan, Naila
    National University of Computer & Emerging Sciences Islamabad, Pakistan.
    Saadatmand, Mehrdad
    Research Institutes of Sweden Västerås, Sweden.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    MBRP: Model-based Requirements Prioritization Using PageRank Algorithm2019In: Asia-Pacific Software Engineering Conference APSEC 2019, 2019, p. 31-38, article id 8945656Conference paper (Refereed)
    Abstract [en]

    Requirements prioritization plays an important role in driving project success during software development. Literature reveals that existing requirements prioritization approaches ignore vital factors such as interdependency between requirements. Existing requirements prioritization approaches are also generally time-consuming and involve substantial manual effort. Besides, these approaches show substantial limitations in terms of the number of requirements under consideration. There is some evidence suggesting that models could have a useful role in the analysis of requirements interdependency and their visualization, contributing towards the improvement of the overall requirements prioritization process. However, to date, just a handful of studies are focused on model-based strategies for requirements prioritization, considering only conflict-free functional requirements. This paper uses a meta-model-based approach to help the requirements analyst to model the requirements, stakeholders, and inter-dependencies between requirements. The model instance is then processed by our modified PageRank algorithm to prioritize the given requirements. An experiment was conducted, comparing our modified PageRank algorithm’s efficiency and accuracy with five existing requirements prioritization methods. Besides, we also compared our results with a baseline prioritized list of 104 requirements prepared by 28 graduate students. Our results show that our modified PageRank algorithm was able to prioritize the requirements more effectively and efficiently than the other prioritization methods.

  • 4.
    Abbas, Muhammad
    et al.
    RISE Research Institutes of Sweden, Västerås, Sweden.
    Jongeling, Robbert
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Lindskog, Claes
    Bombardier Transportation AB, Sweden.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, Mehrdad
    RISE Research Institutes of Sweden, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Product Line Adoption in Industry: An Experience Report from the Railway Domain2020In: ACM International Conference Proceeding Series, 2020, Vol. F164267-A, p. 130-141Conference paper (Refereed)
    Abstract [en]

    The software system controlling a train is typically deployed on various hardware architectures and is required to process various signals across those deployments. Increases of such customization scenarios, as well as the needed adherence of the software to various safety standards in different application domains, has led to the adoption of product line engineering within the railway domain. This paper explores the current state-of-practice of software product line development within a team developing industrial embedded software for a train propulsion control system. Evidence is collected by means of a focus group session with several engineers and through inspection of archival data. We report several benefits and challenges experienced during product line adoption and deployment. Furthermore, we identify and discuss research opportunities, focusing in particular on the areas of product line evolution and test automation.

  • 5.
    Abbas, Muhammad
    et al.
    RISE Research Institutes of Sweden.
    Rauf, Abdul
    RISE Research Institute of Sweden.
    Saadatmand, Mehrdad
    RISE Research Institute of Sweden.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Keywords-based test categorization for Extra-Functional Properties2020In: IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020, IEEE, 2020, p. 153-156, article id 9156019Conference paper (Refereed)
    Abstract [en]

    Categorizing existing test specifications can provide insights on coverage of the test suite to extra-functional properties. Manual approaches for test categorization can be time-consuming and prone to error. In this short paper, we propose a semi-automated approach for semantic keywords-based textual test categorization for extra-functional properties. The approach is the first step towards coverage-based test case selection based on extra-functional properties. We report a preliminary evaluation of industrial data for test categorization for safety aspects. Results show that keyword-based approaches can be used to categorize tests for extra-functional properties and can be improved by considering contextual information of keywords.

  • 6.
    Abbas, Muhammad
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, Mehrdad
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Lindskog, Claes
    Bombardier Transportation AB, Sweden.
    Automated Reuse Recommendation of Product Line Assets based on Natural Language Requirements2020In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020, Vol. 12541, p. 173-189Conference paper (Refereed)
    Abstract [en]

    Software product lines (SPLs) are based on reuse rationale to aid quick and quality delivery of complex products at scale. Deriving a new product from a product line requires reuse analysis to avoid redundancy and support a high degree of assets reuse. In this paper, we propose and evaluate automated support for recommending SPL assets that can be reused to realize new customer requirements. Using the existing customer requirements as input, the approach applies natural language processing and clustering to generate reuse recommendations for unseen customer requirements in new projects. The approach is evaluated both quantitatively and qualitatively in the railway industry. Results show that our approach can recommend reuse with 74% accuracy and 57.4% exact match. The evaluation further indicates that the recommendations are relevant to engineers and can support the product derivation and feasibility analysis phase of the projects. The results encourage further study on automated reuse analysis on other levels of abstractions.

  • 7.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Runtime Verification based Concurrency Bug Detector for FreeRTOS Embedded Software2018In: Proceedings - 17th International Symposium on Parallel and Distributed Computing, ISPDC 2018, 2018, p. 172-179, article id 8452035Conference paper (Refereed)
    Download full text (pdf)
    fulltext
  • 8.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.
    Towards Classification of Concurrency Bugs Based on Observable Properties2015In: Proceedings - 1st International Workshop on Complex Faults and Failures in Large Software Systems, COUFLESS 2015, 2015, p. 41-47Conference paper (Refereed)
    Abstract [en]

    In software engineering, classification is a way to find an organized structure of knowledge about objects. Classification serves to investigate the relationship between the items to be classified, and can be used to identify the current gaps in the field. In many cases users are able to order and relate objects by fitting them in a category. This paper presents initial work on a taxonomy for classification of errors (bugs) related to concurrent execution of application level software threads. By classifying concurrency bugs based on their corresponding observable properties, this research aims to examine and structure the state of the art in this field, as well as to provide practitioner support for testing and debugging of concurrent software. We also show how the proposed classification, and the different classes of bugs, relates to the state of the art in the field by providing a mapping of the classification to a number of recently published papers in the software engineering field.

  • 9.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, Mehrdad
    SICS Swedish ICT, Västerås, Sweden.
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Model for Systematic Monitoring and Debugging of Starvation Bugs in Multicore Software2016In: 2016 ASE Workshop on Specification, Comprehension, Testing and Debugging of Concurrent Programs SCTDCP2016, 2016Conference paper (Refereed)
    Abstract [en]

    With the development of multicore hardware, concurrent, parallel and multicore software are becoming increasingly popular. Software companies are spending a huge amount of time and resources to nd and debug the bugs. Among all types of software bugs, concurrency bugs are also important and troublesome. This type of bugs is increasingly becoming an issue particularly due to the growing prevalence of multicore hardware. In this position paper, we propose a model for monitoring and debugging Starvation bugs as a type of concurrency bugs in multicore software. The model is composed into three phases: monitoring, detecting and debugging. The monitoring phase can support detecting phase by storing collected data from the system execution. The detecting phase can support debugging phase by comparing the stored data with starvation bug's properties, and the debugging phase can help in reproducing and removing the Starvation bug from multicore software. Our intention is that our model is the basis for developing tool(s) to enable solving Starvation bugs in software for multicore platforms.

  • 10.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, S.
    Ericsson AB, Kista, Sweden.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Runtime Verification Tool for Detecting Concurrency Bugs in FreeRTOS Embedded Software2018In: Proceedings - 17th International Symposium on Parallel and Distributed Computing, ISPDC 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 172-179, article id 8452035Conference paper (Refereed)
    Abstract [en]

    This article presents a runtime verification tool for embedded software executing under the open source real-time operating system FreeRTOS. The tool detects and diagnoses concurrency bugs such as deadlock, starvation, and suspension based-locking. The tool finds concurrency bugs at runtime without debugging and tracing the source code. The tool uses the Tracealyzer tool for logging relevant events. Analysing the logs, our tool can detect the concurrency bugs by applying algorithms for diagnosing each concurrency bug type individually. In this paper, we present the implementation of the tool, as well as its functional architecture, together with illustration of its use. The tool can be used during program testing to gain interesting information about embedded software executions. We present initial results of running the tool on some classical bug examples running on an AVR 32-bit board SAM4S. 

  • 11.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, S.
    Ericsson AB, Stockholm, Sweden.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Concurrency bugs in open source software: a case study2017In: Journal of Internet Services and Applications, ISSN 1867-4828, E-ISSN 1869-0238, Vol. 8, no 1, article id 4Article in journal (Refereed)
    Abstract [en]

    Concurrent programming puts demands on software debugging and testing, as concurrent software may exhibit problems not present in sequential software, e.g., deadlocks and race conditions. In aiming to increase efficiency and effectiveness of debugging and bug-fixing for concurrent software, a deep understanding of concurrency bugs, their frequency and fixing-times would be helpful. Similarly, to design effective tools and techniques for testing and debugging concurrent software, understanding the differences between non-concurrency and concurrency bugs in real-word software would be useful. This paper presents an empirical study focusing on understanding the differences and similarities between concurrency bugs and other bugs, as well as the differences among various concurrency bug types in terms of their severity and their fixing time, and reproducibility. Our basis is a comprehensive analysis of bug reports covering several generations of five open source software projects. The analysis involves a total of 11860 bug reports from the last decade, including 351 reports related to concurrency bugs. We found that concurrency bugs are different from other bugs in terms of their fixing time and severity while they are similar in terms of reproducibility. Our findings shed light on concurrency bugs and could thereby influence future design and development of concurrent software, their debugging and testing, as well as related tools.

  • 12.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Ericsson AB, Kista, Sweden .
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afza, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    10 Years of research on debugging concurrent and multicore software: a systematic mapping study2017In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 25, no 1, p. 49-82Article in journal (Refereed)
    Abstract [en]

    Debugging – the process of identifying, localizing and fixing bugs – is a key activity in software development. Due to issues such as non-determinism and difficulties of reproducing failures, debugging concurrent software is significantly more challenging than debugging sequential software. A number of methods, models and tools for debugging concurrent and multicore software have been proposed, but the body of work partially lacks a common terminology and a more recent view of the problems to solve. This suggests the need for a classification, and an up-to-date comprehensive overview of the area. 

    This paper presents the results of a systematic mapping study in the field of debugging of concurrent and multicore software in the last decade (2005– 2014). The study is guided by two objectives: (1) to summarize the recent publication trends and (2) to clarify current research gaps in the field.

    Through a multi-stage selection process, we identified 145 relevant papers. Based on these, we summarize the publication trend in the field by showing distribution of publications with respect to year , publication venues , representation of academia and industry , and active research institutes . We also identify research gaps in the field based on attributes such as types of concurrency bugs, types of debugging processes , types of research  and research contributions.

    The main observations from the study are that during the years 2005–2014: (1) there is no focal conference or venue to publish papers in this area, hence a large variety of conferences and journal venues (90) are used to publish relevant papers in this area; (2) in terms of publication contribution, academia was more active in this area than industry; (3) most publications in the field address the data race bug; (4) bug identification is the most common stage of debugging addressed by articles in the period; (5) there are six types of research approaches found, with solution proposals being the most common one; and (6) the published papers essentially focus on four different types of contributions, with ”methods” being the type most common one.

    We can further conclude that there is still quite a number of aspects that are not sufficiently covered in the field, most notably including (1) exploring correction  and fixing bugs  in terms of debugging process; (2) order violation, suspension  and starvation  in terms of concurrency bugs; (3) validation and evaluation research  in the matter of research type; (4) metric  in terms of research contribution. It is clear that the concurrent, parallel and multicore software community needs broader studies in debugging.This systematic mapping study can help direct such efforts.

  • 13.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Paul Enoiu, Eduard
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Study on Concurrency Bugs in an Open Source Software2016In: IFIP Advances in Information and Communication Technology, vol. 472, 2016, Vol. 472, p. 16-31Conference paper (Refereed)
    Abstract [en]

    Concurrent programming puts demands on software debugging and testing, as concurrent software may exhibit problems not present in sequential software, e.g., deadlocks and race conditions. In aiming to increase efficiency and effectiveness of debugging and bug-fixing for concurrent software, a deep understanding of concurrency bugs, their frequency and fixingtimes would be helpful. Similarly, to design effective tools and techniques for testing and debugging concurrent software understanding the differences between non-concurrency and concurrency bugs in real-word software would be useful.

  • 14.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Runtime Verification for Detecting Suspension Bugs in Multicore and Parallel Software2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017, 2017, p. 77-80Conference paper (Refereed)
    Abstract [en]

    Multicore hardware development increases the popularity of parallel and multicore software, while testing and debugging the software become more difficult, frustrating and costly. Among all types of software bugs, concurrency bugs are both important and troublesome. This type of bugs is increasingly becoming an issue, particularly due to the growing prevalence of multicore hardware. Suspension-based-locking bug is one type of concurrency bugs. This position paper proposes a model based on runtime verification and reflection technique in the context of multicore and parallel software to monitor and detect suspension-based-locking bugs. The model is not only able to detect faults, but also diagnose and even repair them. The model is composed of four layers: Logging, Monitoring, Suspension Bug Diagnosis and Mitigation. The logging layer will observe the events and save them into a file system. The monitoring layer will detect the presents of bugs in the software. The suspension bug diagnosis will identify Suspension bugs by comparing the captured data with the suspension bug properties. Finally, the mitigation layer will reconfigure the software to mitigate the suspension bugs. A functional architecture of a runtime verification tool is also proposed in this paper. This architecture is based on the proposed model and is comprised of different modules. 

  • 15.
    Bilic, Damir
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Carlson, Jan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Wallin, Peter
    RISE SICS Västerås, Sweden.
    Detecting Inconsistencies in Annotated Product Line Models2020In: ACM International Conference Proceeding Series, 2020, Vol. F164267-A, p. 252-262, article id 20Conference paper (Refereed)
    Abstract [en]

    Model-based product line engineering applies the reuse practices from product line engineering with graphical modeling for the specification of software intensive systems. Variability is usually described in separate variability models, while the implementation of the variable systems is specified in system models that use modeling languages such as SysML. Most of the SysML modeling tools with variability support, implement the annotation-based modeling approach. Annotated product line models tend to be error-prone since the modeler implicitly describes every possible variant in a single system model.To identifying variability-related inconsistencies, in this paper, we firstly define restrictions on the use of SysML for annotative modeling in order to avoid situations where resulting instances of the annotated model may contain ambiguous model constructs. Secondly, inter-feature constraints are extracted from the annotated model, based on relations between elements that are annotated with features. By analyzing the constraints, we can identify if the combined variability- and system model can result in incorrect or ambiguous instances. The evaluation of our prototype implementation shows the potential of our approach by identifying inconsistencies in the product line model of our industrial partner which went undetected through several iterations of the model.

  • 16.
    Bilic, Damir
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Wallin, Peter
    RISE SICS Västerås, Sweden.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Amlinger, C.
    Volvo Construction Equipment, Eskilstuna, Sweden.
    Barkah, D.
    Volvo Construction Equipment, Eskilstuna, Sweden.
    Towards a Model-Driven Product Line Engineering Process - An Industrial Case Study2020In: ACM International Conference Proceeding Series, NEW YORK: ASSOC COMPUTING MACHINERY , 2020, article id 3385043Conference paper (Refereed)
    Abstract [en]

    Many organizations developing software-intensive systems face challenges with high product complexity and large numbers of variants. In order to effectively maintain and develop these product variants, Product-Line Engineering methods are often considered, while Model-based Systems Engineering practices are commonly utilized to tackle product complexity. In this paper, we report on an industrial case study concerning the ongoing adoption of Product Line Engineering in the Model-based Systems Engineering environment at Volvo Construction Equipment (Volvo CE) in Sweden. In the study, we identify and define a Product Line Engineering process that is aligned with Model-based Systems Engineering activities at the engines control department of Volvo CE. Furthermore, we discuss the implications of the migration from the current development process to a Model-based Product Line Engineering-oriented process. This process, and its implications, are derived by conducting and analyzing interviews with Volvo CE employees, inspecting artifacts and documents, and by means of participant observation. Based on the results of a first system model iteration, we were able to document how Model-based Systems Engineering and variability modeling will affect development activities, work products and stakeholders of the work products.

  • 17.
    Bilic, Damir
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Wallin, Peter
    RISE SICS, Västerås, Sweden.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. ES (Embedded Systems).
    Amlinger, Christoffer
    Volvo CE, Eskilstuna, Sweden.
    Model-Based Product Line Engineering in an Industrial Automotive Context: An Exploratory Case Study2018In: 1st Intl. Workshop on Variability and Evolution of Software-intensive Systems VariVolution'18, 2018Conference paper (Refereed)
    Abstract [en]

    Product Line Engineering is an approach to reuse assets of complex systems by taking advantage of commonalities between product families. Reuse within complex systems usually means reuse of artifacts from different engineering domains such as mechanical, electronics and software engineering. Model-based systems engineering is becoming a standard for systems engineering and collaboration within different domains. This paper presents an exploratory case study on initial efforts of adopting Product Line Engineering practices within the model-based systems engineering process at Volvo Construction Equipment (Volvo CE), Sweden. We have used SysML to create overloaded models of the engine systems at Volvo CE. The variability within the engine systems was captured by using the Orthogonal Variability Modeling language. The case study has shown us that overloaded SysML models tend to become complex even on small scale systems, which in turn makes scalability of the approach a major challenge. For successful reuse and to, possibly, tackle scalability, it is necessary to have a database of reusable assets from which product variants can be derived.

  • 18.
    Brahneborg, Daniel
    et al.
    Infoflex Connect AB, Stockholm, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Björkman, Mats
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Round-Trip Time Anomaly Detection2018In: ICPE '18 Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering, 2018, p. 107-114Conference paper (Refereed)
    Abstract [en]

    Mobile text messages (SMS) are sometimes used for authentication, which requires short and reliable delivery times. The observed round-trip times when sending an SMS message provide valuable information on the quality of the connection. In this industry paper, we propose a method for detecting round-trip time anomalies, where the exact distribution is unknown, the variance is several orders of magnitude, and there are lots of shorter spikes that should be ignored. In particular, we show that using an adaption of Double Seasonal Exponential Smoothing to reduce the content dependent variations, followed by the Remedian to find short-term and long-term medians, successfully identifies larger groups of outliers. As training data for our method we use log files from a live SMS gateway. In order to verify the effectiveness of our approach, we utilize simulated data. Our contributions are a description on how to isolate content dependent variations, and the sequence of steps to find significant anomalies in big data.

  • 19.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Krasteva, Iva
    Mälardalen University, School of Innovation, Design and Engineering.
    Land, Rikard
    Mälardalen University, School of Innovation, Design and Engineering.
    Sajeev, Abdulkadir
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    A Survey on Industrial Software Engineering2009In: AGILE PROCESSES IN SOFTWARE ENGINEERING AND EXTREME PROGRAMMING, Springer, 2009, p. 240-241Chapter in book (Refereed)
    Abstract [en]

    In this paper, we present on-going work on data collected by a questionnaire surveying process practices, preferences, and methods in industrial software engineering.

  • 20.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Krasteva, Iva
    Mälardalen University, School of Innovation, Design and Engineering. Sofia University.
    Land, Rikard
    Mälardalen University, School of Innovation, Design and Engineering.
    Sajeev, Abdulkadir
    Mälardalen University, School of Innovation, Design and Engineering. University of New England, Australia.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    An Industrial Survey on Software Process Practices, Preferences and Methods2009Report (Other academic)
    Abstract [en]

    This report presents the questionnaire questions and responses of a web-based survey conducted during early spring 2009. It was intended to survey current processes, practices, and methods in the software industry. The report contains no analysis of the data, but is intended to be a reference for other publications, both by ourselves, but by making the data publicly available, also by others.

  • 21.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Quality of Testing in Test Driven Development2012In: 2012 8th International Conference on the Quality of Information and Communications Technology, QUATIC 2012: Proceedings, IEEE , 2012, p. 266-271Conference paper (Refereed)
    Abstract [en]

    Test-driven development is an essential part of eXtreme Programming approach with the preference of being followed in other Agile methods as well. For several years, researchers are performing empirical investigations to evaluate quality improvements in the resulting code when test-driven development is being used. However, very little had been reported into investigating the quality of the testing performed in conjunction with test-driven development. In this paper we present results from an experiment specifically designed to evaluate the quality of test cases created by developers who used the test-first and the traditional test-last approaches. On an average, the quality of testing in test-driven development was almost the same as the quality of testing using test-last approach. However, detailed analysis of test cases, created by test-driven development group, revealed that 29% of test cases were "negative" test cases (based on non-specified requirements) but contributing as much as 65% to the overall tests quality score of test-first developers. We are currently investigating the possibility of extending test-driven development to facilitate non-specified requirements to a higher extent and thus minimise the impact of a potentially inherent effect of positive test bias.

  • 22.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    TDDHQ: Achieving Higher Quality Testing in Test Driven Development2013In: Euromicro Conference on Software Engineering and Advanced Applications, 2013, p. 33-36Conference paper (Refereed)
    Abstract [en]

    Test driven development (TDD) appears not to be immune to positive test bias effects, as we observed in several empirical studies. In these studies, developers created a significantly larger set of positive tests, but at the same time the number of defects detected with negative tests is significantly higher than those detected by positive ones. In this paper we propose the concept of TDDHQ which is aimed at achieving higher quality of testing in TDD by augmenting the standard TDD with suitable test design techniques. To exemplify this concept, we present combining equivalence partitioning test design technique together with the TDD, for the purpose of improving design of test cases. Initial evaluation of this approach showed a noticeable improvement in the quality of test cases created by developers utilising TDDHQ approach.

  • 23.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Shukla, Rakesh
    Infosys Ltd., India.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Effects of Negative Testing on TDD: An Industrial Experiment2013In: Agile Processes in Software Engineering and Extreme Programming: 14th International Conference, XP 2013, Vienna, Austria, June 3-7, 2013. Proceedings, Springer , 2013, p. 91-105Chapter in book (Refereed)
    Abstract [en]

    In our recent academic experiments, an existence of positive test bias, that is lack of negative test cases, was identified when a test driven development approach was used. At the same time, when defect detecting ability of individual test cases was calculated, it was noted that the probability of a negative test case to detect a defect was substantially higher than that of a positive test case. The goal of this study is to investigate the existence of positive test bias in test driven development within an industrial context, and measure defect detecting ability of both positive and negative test cases. An industrial experiment was conducted at Infosys Ltd. India, whose employees voluntarily signed up to participate in the study and were randomly assigned to groups utilizing test driven development, test driven development with negative testing, and test last development. Source code and test cases created by each participant during the study were collected and analysed. The collected data indicate a statistically significant difference between the number of positive and negative test cases created by industrial participants, confirming the existence of positive test bias. The difference in defect detecting ability of positive and negative test cases is also statistically significant. As a result, similarly to our previous academic study, 29% of all test cases were negative, contributing by revealing as much as 71% of all the defects found by all test cases. With this industrial experiment, we confirmed the existence of a positive test bias in an industrial context, as well as significantly higher defect detecting ability of negative test cases.

  • 24.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering.
    An Industrial Survey on Contemporary Aspects of Software Testing2010In: ICST 2010 - 3rd International Conference on Software Testing, Verification and Validation, Paris, France, 2010, p. 393-401Conference paper (Refereed)
    Abstract [en]

    Software testing is a major source of expense in software projects and a proper testing process is a critical ingredient in the cost-efficient development of high-quality software. Contemporary aspects, such as the introduction of a more lightweight process, trends towards distributed development, and the rapid increase of software in embedded and safety-critical systems, challenge the testing process in unexpected manners. To our knowledge, there are very few studies focusing on these aspects in relation to testing as perceived by different contributors in the software development process. This paper qualitatively and quantitatively analyses data from an industrial questionnaire survey, with a focus on current practices and preferences on contemporary aspects of software testing. Specifically, the analysis focuses on perceptions of the software testing process in different categories of respondents. Categorization of respondents is based on safety-criticality, agility, distribution of development, and application domain. While confirming some of the commonly acknowledged facts, our findings also reveal notable discrepancies between preferred and actual testing practices. We believe continued research efforts are essential to provide guidelines in the adaptation of the testing process to take care of these discrepancies, thus improving the quality and efficiency of the software development.

  • 25.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering.
    Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review2011In: 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011; Berlin; 21 March 2011 through 25 March 2011, 2011, p. 337-346Conference paper (Refereed)
    Abstract [en]

    Test driven development (TDD) is one of the basic practices of agile software development and both academia and practitioners claim that TDD, to a certain extent, improves the quality of the code produced by developers. However, recent results suggest that this practice is not followed to the extent preferred by industry. In order to pinpoint specific obstacles limiting its industrial adoption we have conducted a systematic literature review on empirical studies explicitly focusing on TDD as well as indirectly addressing TDD. Our review has identified seven limiting factors viz., increased development time, insufficient TDD experience/knowledge, lack of upfront design, domain and tool specific issues, lack of developer skill in writing test cases, insufficient adherence to TDD protocol, and legacy code. The results of this study is of special importance to the testing community, since it outlines the direction for further detailed scientific investigations as well as highlights the requirement of guidelines to overcome these limiting factors for successful industrial adoption of TDD.

    Download full text (pdf)
    FULLTEXT01
  • 26.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering.
    Impact of Test Design Technique Knowledge on Test Driven Development: A Controlled Experiment2012In: Lecture Notes in Business Information Processing, vol. 111 LNBIP, Springer, 2012, p. 138-152Chapter in book (Refereed)
    Abstract [en]

    Agile development approaches are increasingly being followed and favored by the industry. Test Driven Development (TDD) is a key agile practice and recent research results suggest that the successful adoption of TDD depends on different limiting factors, one of them being insufficient developer testing skills. The goal of this paper is to investigate if developers who are educated on general testing knowledge will be able to utilize TDD more efectively. We conducted a controlled experiment with master students during the course on Software Verification & Validation (V&V) where source code and test cases created by each participant during the labs as well as their answers on a survey questionnaire were collected and analyzed. Descriptive statistics indicate improvements in statement coverage. However, no statistically significant differences could be established between the pre- and post-course groups of students. By qualitative analysis of students’ tests, we noticed a lack of test cases for non-stated requirements ("negative" tests) resulting in a non-detection of bugs. Students did show preference towards TDD in surveys. Although further research is required to fully establish this, we believe that identifying specific testing knowledge which is complementary to the testing skills of a new TDD developer would enable developers to perform their tasks in a more eficient manner.

  • 27.
    Causevic, Adnan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Punnekkat, Sasikumar
    Mälardalen University, School of Innovation, Design and Engineering.
    Test Case Quality in Test Driven Development: A Study Design and a Pilot Experiment2012In: EASE 2012, Proceedings, 2012, p. 223-227Conference paper (Refereed)
    Abstract [en]

    Background: Test driven development, as a side-effect of developing software, will produce a set of accompanied test cases which can protect implemented features during code refactoring. However, recent research results point out that successful adoption of test driven development might be limited by the testing skills of developers using it. Aim: Main goal of this paper is to investigate if there is a difference between the quality of test cases created while using test-first and test-last approaches. Additional goal of this paper is to measure the code quality produced using test-first and test-last approaches. Method: A pilot study was conducted during the master level course on Software Verification & Validation at Mälardalen University. Students were working individually on the problem implementation by being randomly assigned to a test-first or a test-last (control) group. Source code and test cases created by each participant during the study, as well as their answers on a survey questionnaire after the study, were collected and analysed. The quality of the test cases is analysed from three perspectives: (i) code coverage, (ii) mutation score and (iii) the total number of failing assertions. Results: The total number of test cases with failing assertions (test cases revealing an error in the code) was nearly the same for both test-first and test-last groups. This can be interpreted as "test cases created by test-first developers were as good as (or as bad as) test cases created by test-last developers". On the contrary, solutions created by test-first developers had, on average, 27% less failing assertions when compared to solutions created by the test-last group. Conclusions: Though the study provided some interesting observations, it needs to be conducted as a fully controlled experiment with a higher number of participants in order to validate statistical significance of the presented results.

  • 28.
    Derehag, Jesper
    et al.
    Ericsson AB, Gothenburg, Sweden..
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Transitioning Fault Prediction Models to a New Environment2016In: Proceedings - 2016 12th European Dependable Computing Conference, EDCC 2016, 2016, p. 241-248, article id 7780365Conference paper (Refereed)
    Abstract [en]

    We describe the application and evaluation of fault prediction algorithms to a project developed by a Swedish company that transitioned from waterfall to agile development methods. The project used two different version control systems and a separate bug tracking system during its lifetime. The algorithms were originally designed for use on systems implemented with a traditional waterfall process at American companies that maintained their project records in an integrated database system that combined bug recording and version control. We compare the performance of the original prediction model on the American systems to the results obtained in the Swedish environment in both its pre-agile and agile stages. We also consider the impact of additional variables in the model.

  • 29.
    Ekman, M.
    et al.
    Bombardier Transportation Sweden AB, Västerås, Sweden.
    Thane, H.
    Safety Integrity AB, Västerås, Sweden .
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Larsson, S.
    Effective Change AB, Västerås, Sweden .
    Tool qualification for safety related systems2014In: Ada User Journal, ISSN 1381-6551, Vol. 35, no 1, p. 47-54Article in journal (Refereed)
    Abstract [en]

    Tools used in the development of safety related software applications need to be qualified as safe. That is, the tools cannot be allowed to introduce hazardous faults into the application, e.g., a compiler shall not generate dangerous code due to failure of the compiler. In many cases laws and regulations require the product development of safety related applications to comply with industry sector specific safety standards. Examples of such standards include EN50129/50128 for railway applications, ISO/EN13849 for machines with moving parts, DO-178B/C for avionics, or IS026262 for cars. These standards require the use of a rigorous development and maintenance process. The standards are also mainly intended to be used when developing systems from scratch. However, most development and test tools are not developed from scratch according to the rigorous processes of these standards. In order to address this issue, some of the standards provide means for qualifying existing tools as a more lightweight and pragmatic alternative to a regular certification process. In this paper we analyze the concept of these qualification approaches. The result of the analysis in our contribution includes a set of approaches that can be applied individually or as a combination in order to reduce the effort needed for qualifying tools. As a running example we use one of the most flexible but at the same time dangerous, even prohibited, maintenance techniques available: dynamic instrumentation of executing code. With this example, we describe how exceptions in these standards can be utilized in order to qualify a dynamic instrumentation tool with a minimal effort, without following the process of tool certification as defined by the standards.

  • 30.
    Eldh, Sigrid
    et al.
    Mälardalen University, Department of Computer Science and Electronics.
    Hansson, Hans
    Mälardalen University, Department of Computer Science and Electronics.
    Punnekkat, Sasikumar
    Mälardalen University, Department of Computer Science and Electronics.
    Pettersson, Anders
    Mälardalen University, Department of Computer Science and Electronics.
    Sundmark, Daniel
    Mälardalen University, Department of Computer Science and Electronics.
    Framework for Comparing Efficiency, Effectiveness and Applicability of Software Testing Techniques2006In: Proceedings - Testing: Academic and Industrial Conference - Practice and Research Techniques, TAIC PART 2006, 2006, p. 159-170, article id 1691683Conference paper (Refereed)
    Abstract [en]

    Software testing is expensive for the industry, and always constrained by time and effort. Although there is a multitude of test techniques, there are currently no scientifically based guidelines for the selection of appropriate techniques of different domains and contexts. For large complex systems, some techniques are more efficient in finding failures than others and some are easier to apply than others are. From an industrial perspective, it is important to find the most effective and efficient test design technique that is possible to automate and apply. In this paper, we propose an experimental framework for comparison of test techniques with respect to efficiency, effectiveness and applicability. We also plan to evaluate ease of automation, which has not been addressed by previous studies. We highlight some of the problems of evaluating or comparingtest techniques in an objective manner. We describe our planned process for this multi-phase experimental study. This includes presentation of some of the important measurements to be collected with the dual goals of analyzing the properties of the test technique, as well as validating our experimental framework.

  • 31.
    Eldh, Sigrid
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Robustness Testing of Mobile Telecommunication Systems: A Case Study on Industrial Practice and Challenges2012In: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012, 2012, p. 895-900Conference paper (Refereed)
    Abstract [en]

    Context: Robustness relates to the capability of a system to handle internal and external negative situations and disturbances. Robustness testing is the act of subjecting the system under test to such disturbances in a controlled manner. Objective: The objective of this study is to understand how robustness is considered in the development and testing of large-scale telecom systems, to identify the main challenges related to robustness and robustness testing, and to identify potential improvements to the current situation. Method: An exploratory case study of a Swedish telecom company was conducted. Data was collected through inter- views, study of documentation, and participant observation. Results: A number of challenges related to robustness testing was identified. Even though there is a relatively consistent view of robustness within the company, and robustness is considered to be an important quality attribute, there is a lack of explicit focus on robustness in design and requirements analysis. Conclusion: Our conclusion is that the area of robustness test is challenging for large complex systems, where the understanding of how to provoke complex failures and derive a root cause, as well as defining the correct level of robust design in software is not sufficiently explored.

  • 32.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Software Engineering Research Consultant, Västerås, Sweden.
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Swedish Institute of Computer Science, Stockholm, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated Test Generation using Model-Checking: An Industrial Evaluation2016In: International Journal on Software Tools for Technology Transfer, ISSN 1433-2779, E-ISSN 1433-2787, Vol. 18, no 3, p. 335-353Article in journal (Refereed)
    Abstract [en]

    In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model-checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state-space explosion can, however, impact model-checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model-checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox COMPLETETEST can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.

    Download full text (pdf)
    fulltext
  • 33.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Swedish Institute of Computer Science, Kista, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Controlled Experiment in Testing of Safety-Critical Embedded Software2016In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, 2016, p. 1-11Conference paper (Refereed)
    Abstract [en]

    In engineering of safety critical systems, regulatory standards often put requirements on both traceable specification-based testing, and structural coverage on program units. Automated test generation techniques can be used to generate inputs to cover the structural aspects of a program. However, there is no conclusive evidence on how automated test generation compares to manual test design, or how testing based on the program implementation relates to specification-based testing. In this paper, we investigate specification-and implementation-based testing of embedded software written in the IEC 61131-3 language, a programming standard used in many embedded safety critical software systems. Further, we measure the efficiency and effectiveness in terms of fault detection. For this purpose, a controlled experiment was conducted, comparing tests created by a total of twenty-three software engineering master students. The participants worked individually on manually designing and automatically generating tests for two IEC 61131-3 programs. Tests created by the participants in the experiment were collected and analyzed in terms of mutation score, decision coverage, number of tests, and testing duration. We found that, when compared to implementation-based testing, specification-based testing yields significantly more effective tests in terms of the number of faults detected. Specifically, specification-based tests more effectively detect comparison and value replacement type of faults, compared to implementation-based tests. On the other hand, implementation-based automated test generation leads to fewer tests (up to 85% improvement) created in shorter time than the ones manually created based on the specification.

    Download full text (pdf)
    fulltext
  • 34.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Doganay, Kivanc
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    MOS: An Integrated Model-based and Search-based Testing Tool for Function Block Diagrams2013In: 2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering, CMSBSE 2013 - Proceedings, 2013, p. 55-60Conference paper (Refereed)
    Abstract [en]

    In this paper we present a new testing tool for safety critical applications described in Function Block Diagram (FBD) language aimed to support both a model and a search-based approach. Many benefits emerge from this tool, including the ability to automatically generate test suites from an FBD program in order to comply to quality requirements such as component testing and specific coverage measurements. Search-based testing methods are used to generate test data based on executable code rather than the FBD program, alleviating any problems that may arise from the ambiguities that occur while creating FBD programs. Test cases generated by both approaches are executed and used as a way of cross validation. In the current work, we describe the architecture of the tool, its workflow process, and a case study in which the tool has been applied in a real industrial setting to test a train control management system.

    Download full text (pdf)
    fulltext
  • 35.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Feldt, Robert
    Blekinge Institute of Technology, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Mutation-Based Test Generation for PLC Embedded Software using Model Checking2016In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016, Vol. 9976, p. 155-171Conference paper (Refereed)
    Abstract [en]

    Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

    Download full text (pdf)
    fulltext
  • 36.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Comparative Study of Manual and Automated Testing for Industrial Control Software2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, 2017, p. 412-417, article id 7927994Conference paper (Refereed)
    Abstract [en]

    Automated test generation has been suggested as a way of creating tests at a lower cost. Nonetheless, it is not very well studied how such tests compare to manually written ones in terms of cost and effectiveness. This is particularly true for industrial control software, where strict requirements on both specification-based testing and code coverage typically are met with rigorous manual testing. To address this issue, we conducted a case study in which we compared manually and automatically created tests. We used recently developed real-world industrial programs written in the IEC 61131-3, a popular programming language for developing industrial control systems using programmable logic controllers. The results show that automatically generated tests achieve similar code coverage as manually created tests, but in a fraction of the time (an average improvement of roughly 90%). We also found that the use of an automated test generation tool does not result in better fault detection in terms of mutation score compared to manual testing. Specifically, manual tests more effectively detect logical, timer and negation type of faults, compared to automatically generated tests. The results underscore the need to further study how manual testing is performed in industrial practice and the extent to which automated test generation can be used in the development of reliable systems.

    Download full text (pdf)
    fulltext
  • 37.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Comparative Study of Manual and Automated Testing in Industrial Embedded SoftwareManuscript (preprint) (Other (popular science, discussion, etc.))
    Abstract [en]

    Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

    Download full text (pdf)
    fulltext
  • 38.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Model-based test suite generation for function block diagrams using the UPPAAL model checker2013In: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013, 2013, p. 158-167Conference paper (Refereed)
    Abstract [en]

    A method for model-based test generation of safety-critical embedded applications using Programmable Logic Controllers and implemented in a programming language such as Function Block Diagram (FBD) is described. The FBD component model is based on the IEC 1131 standard and it is used primarily for embedded systems, in which timeliness is an important property to be tested. Our method involves the transformation of FBD programs with timed annotations into timed automata models which are used to automatically generate test suites. Specifically we demonstrate how to use model transformation for formalization and model checking of FBD programs using the UPPAAL tool. Many benefits emerge from this method, including the ability to automatically generate test suites from a formal model in order to ensure compliance to strict quality requirements including unit testing and specific coverage measurements. The approach is experimentally assessed on a train control system in terms of consumed resources.

    Download full text (pdf)
    fulltext
  • 39.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Using Logic Coverage to Improve Testing Function Block Diagrams2013In: Testing Software and Systems: Lecture Notes in Computer Science, Volume 8254, Springer Berlin Heidelberg , 2013, p. 1-16Chapter in book (Refereed)
    Abstract [en]

    In model-driven development, testers are often focusing on functional model-level testing, enabling verification of design models against their specifications. In addition, in safety-critical software development, testers are required to show that tests cover the structure of the implementation. Testing cost and time savings could be achieved if the process of deriving test cases for logic coverage is automated and provided test cases are ready to be executed. The logic coverage artifacts, i.e., predicates and clauses, are required for different logic coverage, e.g., MC/DC. One way of dealing with test case generation for ensuring logic coverage is to approach it as a model-checking problem, such that model-checking tools automatically create test cases. We show how logic coverage criteria can be formalized and used by a model-checker to provide test cases for ensuring this coverage on safety-critical software described in the Function Block Diagram programming language. Based on our experiments, this approach, supported by a tool chain, is an applicable and useful way of generating test cases for covering Function Block Diagrams.

    Download full text (pdf)
    fulltext
  • 40.
    Fattouh, Anas
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Innovation and Product Realisation.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Real-Time Optimization Model for Production Planning in Quarry Sites2021In: 2021 IEEE 8th International Conference on Industrial Engineering and Applications, ICIEA 2021, Institute of Electrical and Electronics Engineers Inc. , 2021, p. 32-36, article id 9436725Conference paper (Refereed)
    Abstract [en]

    A quarry site is used to produce gravel with different granularities based on the customer's needs. The production process consists of different stages that need to be planned in order to achieve high productivity and low cost. During the operation of the quarry site, the operating parameters change, and the site productivity could be affected, therefore, a replanning of the production is needed. This paper presents a real-time optimization model for production planning in a quarry site. The optimization model consists of three modules, a productivity monitoring module, a simulation model, and an optimization algorithm. The productivity monitoring module produces the site productivity based on the current operating parameters of the quarry site. Once the site productivity reaches a specific threshold, the optimization algorithm is run on an updated simulation model to find the optimal values of the site parameters. The proposed real-time optimization model is applied to a real quarry site simulated on a high-fidelity simulator. The case study shows that the proposed optimization model can effectively improve the productivity of the quarry site and hence reduces the wastes and improve environmental sustainability.

  • 41.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Exploring Test Overlap in System Integration: An Industrial Case Study2016In: 42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016, 2016, p. 303-312Conference paper (Refereed)
    Abstract [en]

    Tougher safety regulations, global competition and ever increasing complexity of embedded software puts extensive pressure on the effectiveness of the software testing procedures. Previous studies have found that there exist overlaps (i.e., multiple instances of highly similar test cases) and even redundancies in the software testing process. Such overlap has been found between versions, variants and integration levels, but primarily at unit test level. Given large embedded systems involving many subsystems, does overlap exist within the system integration testing as well? In this paper, we present an industrial case study, aiming to a) evaluate if there exist test overlaps within the given context, b) if so, investigate how these overlaps are distributed, and c) find ways of reducing test effort by investigating how the knowledge of overlaps and their distribution may be used for finding candidate test cases for automation, maintenance or even removal. We have studied manual test cases, written in natural language, at a large vehicular manufacturer in Sweden. In particular, we have collected and analyzed test cases from the system integration testing levels of four different projects of a vehicle control management system. Using a similarity function, we evaluate if any overlaps between test cases exist, and where. We found that overlaps do exist within the system integration level, particularly in the form of partial test step sequences. However, very few test cases overlapped in their entirety. Some candidates for test step automation and update propagation were identified, but none for easy removal.

  • 42.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Gustafsson, T.
    Scania CV AB, Sweden.
    Kobetski, A.
    RISE SICS AB, Sweden.
    From natural language requirements to passive test cases using guarded assertions2018In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 470-481Conference paper (Refereed)
    Abstract [en]

    In large-scale embedded system development, requirements are often expressed in natural language. Translating these requirements to executable test cases, while keeping the test cases and requirements aligned, is a challenging task. While such a transformation typically requires extensive domain knowledge, we show that a systematic process in combination with passive testing would facilitate the translation as well as linking the requirements to tests. Passive testing approaches observe the behavior of the system and test their correctness without interfering with the normal behavior. We use a specific approach to passive testing: guarded assertions (G/A). This paper presents a method for transforming system requirements expressed in natural language into G/As. We further present a proof of concept evaluation, performed at Bombardier Transportation Sweden AB, in which we show how the process would be used, together with practical advice of the reasoning behind the translation steps.

    Download full text (pdf)
    fulltext
  • 43.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pasqualina, Potena
    RISE.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    RISE.
    Similarity-Based Prioritization of Test Case Automation2018In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 26, no 4, p. 1421-1449Article in journal (Refereed)
    Abstract [en]

    The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.

    Download full text (pdf)
    fulltext
  • 44.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study2015In: The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, 2015, p. 317-324Conference paper (Refereed)
    Abstract [en]

    Vertical test reuse refers to the the reuse of test cases or other test artifacts over different integration levels in the software or system engineering process. Vertical test reuse has previously been proposed for reducing test effort and improving test effectiveness, particularly for embedded system development. The goal of this study is to provide an overview of the state of the art in the field of vertical test reuse for embedded system development. For this purpose, a systematic mapping study has been performed, identifying 11 papers on vertical test reuse for embedded systems. The primary result from the mapping is a classification of published work on vertical test reuse in the embedded system domain, covering motivations for reuse, reuse techniques, test levels and reusable test artifacts considered, and to what extent the effects of reuse have been evaluated.

  • 45.
    Gustafsson, Thomas
    et al.
    Scania CV, Sweden.
    Skoglund, Mats
    SICS Swedish ICT, Sweden.
    Kobetski, Avenir
    SICS Swedish ICT, Sweden.
    Sundmark, Daniel
    SICS Swedish ICT, Sweden.
    Automotive System Testing by Independent Guarded Assertions2015In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings, 2015, Vol. Article number 7107474Conference paper (Refereed)
    Abstract [en]

    Testing is a key activity in industry to verify and validate products before they reach end customers. In hardwarein- the-loop system-level verification of automotive systems, testing is often performed using sequential execution of test scripts, each containing a mix of stimuli and assertions. In this paper, we propose and study an alternative approach for automated system-level testing automotive systems. In our approach, assertion-only test scripts and one (or several) stimulionly script(s), execute concurrently on the test driver. By separating the stimuli from the assertions, with each assertion independently determining when the system under test shall be verified, we seek to achieve three things: 1) tests that better represent real-world handling of the product, 2) reduced test execution time, and 3) increased defect detection. In addition to describing our proposed approach in detail, we provide experimental results from an industrial case study evaluating the approach in an automotive system test environment.

  • 46.
    Hansson, Hans
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Nolte, Thomas
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sjödin, Mikael
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Real-Time in Networked Embedded Systems2018In: Embedded Systems Handbook: Second Edition, CRC Press , 2018, p. 1-1-1-46Chapter in book (Other academic)
  • 47.
    Hansson, Hans
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Sjödin, Mikael
    Mälardalen University, School of Innovation, Design and Engineering.
    Nolte, Thomas
    Mälardalen University, School of Innovation, Design and Engineering.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering.
    Real-Time in Networked Embedded Systems2009In: Networked Embedded Systems, CRC Press, Taylor & Francis Group , 2009Chapter in book (Other academic)
  • 48.
    Karlsson, Stefan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    ABB AB, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automatic Property-based Testing of GraphQL APIs2021In: Proceedings - 2021 IEEE/ACM International Conference on Automation of Software Test, AST 2021, 2021Conference paper (Refereed)
    Abstract [en]

    In recent years, GraphQL has become a popular way to expose web APIs. With its raise of adoption in industry, the quality of GraphQL APIs must be also assessed, as with any part of a software system, and preferably in an automated manner. However, there is currently a lack of methods to automatically generate tests to exercise GraphQL APIs.

    In this paper, we propose a method for automatically producing GraphQL queries to test GraphQL APIs. This is achieved using a property-based approach to create a generator for queries based on the GraphQL schema of the system under test.

    Our evaluation on a real world software system shows that this approach is both effective, in terms of finding real bugs, and efficient, as a complete schema can be covered in seconds. In addition, we evaluate the fault finding capability of the method when seeding known faults. 73% of the seeded faults were found, with room for improvements with regards to domain-specific behavior, a common oracle challenge in automatic test generation.

  • 49.
    Karlsson, Stefan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Abb Ab, Västerås, Sweden.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Abb Ab, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Abb Ab, Västerås, Sweden.
    QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIs2020In: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation, ICST 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 131-141, article id 9159071Conference paper (Refereed)
    Abstract [en]

    RESTful APIs are an increasingly common way to expose software systems functionality and it is therefore of high interest to find methods to automatically test and verify such APIs. To lower the barrier for industry adoption, such methods need to be straightforward to use with a low effort. This paper introduces a method to explore the behaviour of a RESTful API. This is done by using automatic property-based tests produced from OpenAPI documents that describe the REST API under test. We describe how this method creates artifacts that can be leveraged both as property-based test generators and as a source of validation for results (i.e., as test oracles). Experimental results, on both industrial and open source services, indicate how this approach is a low effort way of finding real faults. Furthermore, it supports building additional knowledge about the system under test by automatically exposing misalignment of specification and implementation. Since the tests are generated from the OpenAPI document this method automatically evolves test cases as the REST API evolves. 

  • 50.
    Karlsson, Stefan
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. ABB AB, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Larsson, Mårten
    ABB AB, Västerås, Sweden.
    Model-based Automated Testing of Mobile Applications: An Industrial Case Study2021In: Proceedings: 2021 IEEE 14th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2021, 2021, p. 130-137Conference paper (Refereed)
    Abstract [en]

    Automatic testing of mobile applications has been a well-researched area in recent years. However, testing in industry is still a very manual practice, as research results have not been fully transferred and adopted. Considering mobile applications, manual testing has the additional burden of adequate testing posed by a large number of available devices and different configurations, as well as the maintenance and setup of such devices.

    In this paper, we propose and evaluate the use of a model-based test generation approach, where generated tests are executed on a set of cloud-hosted real mobile devices. By using a model-based approach we generate dynamic, less brittle, and implementation simple test cases. The test execution on multiple real devices with different configurations increase the confidence in the implementation of the system under test. Our evaluation shows that the used approach produces a high coverage of the parts of the application related to user interactions. Nevertheless, the inclusion of external services in test generation is required in order to additionally increase the coverage of the complete application. Furthermore, we present the lessons learned while transferring and implementing this approach in an industrial context and applying it to the real product.

12 1 - 50 of 90
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf