mdh.sePublications
Change search
Refine search result
12 1 - 50 of 62
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Ericsson AB, Kista, Sweden .
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afza, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    10 Years of research on debugging concurrent and multicore software: a systematic mapping study2017In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 25, no 1, p. 49-82Article in journal (Refereed)
    Abstract [en]

    Debugging – the process of identifying, localizing and fixing bugs – is a key activity in software development. Due to issues such as non-determinism and difficulties of reproducing failures, debugging concurrent software is significantly more challenging than debugging sequential software. A number of methods, models and tools for debugging concurrent and multicore software have been proposed, but the body of work partially lacks a common terminology and a more recent view of the problems to solve. This suggests the need for a classification, and an up-to-date comprehensive overview of the area. 

    This paper presents the results of a systematic mapping study in the field of debugging of concurrent and multicore software in the last decade (2005– 2014). The study is guided by two objectives: (1) to summarize the recent publication trends and (2) to clarify current research gaps in the field.

    Through a multi-stage selection process, we identified 145 relevant papers. Based on these, we summarize the publication trend in the field by showing distribution of publications with respect to year , publication venues , representation of academia and industry , and active research institutes . We also identify research gaps in the field based on attributes such as types of concurrency bugs, types of debugging processes , types of research  and research contributions.

    The main observations from the study are that during the years 2005–2014: (1) there is no focal conference or venue to publish papers in this area, hence a large variety of conferences and journal venues (90) are used to publish relevant papers in this area; (2) in terms of publication contribution, academia was more active in this area than industry; (3) most publications in the field address the data race bug; (4) bug identification is the most common stage of debugging addressed by articles in the period; (5) there are six types of research approaches found, with solution proposals being the most common one; and (6) the published papers essentially focus on four different types of contributions, with ”methods” being the type most common one.

    We can further conclude that there is still quite a number of aspects that are not sufficiently covered in the field, most notably including (1) exploring correction  and fixing bugs  in terms of debugging process; (2) order violation, suspension  and starvation  in terms of concurrency bugs; (3) validation and evaluation research  in the matter of research type; (4) metric  in terms of research contribution. It is clear that the concurrent, parallel and multicore software community needs broader studies in debugging.This systematic mapping study can help direct such efforts.

  • 2.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering. Embedded Systems.
    Search-based approaches to software fault prediction and software testing2009Licentiate thesis, monograph (Other academic)
    Abstract [en]

    Software verification and validation activities are essential for software quality but also constitute a large part of software development costs. Therefore efficient and cost effective software verification and validation activities are both a priority and a necessity considering the pressure to decrease time-to-market and intense competition faced by many, if not all, companies today. It is then perhaps not unexpected that decisions related to software quality, when to stop testing, testing schedule and testing resource allocation needs to be as accurate as possible. This thesis investigates the application of search-based techniques within two activities of software verification and validation: Software fault prediction and software testing for non-functional system properties. Software fault prediction modeling can provide support for making important decisions as outlined above. In this thesis we empirically evaluate symbolic regression using genetic programming (a search-based technique) as a potential method for software fault predictions. Using data sets from both industrial and open-source software, the strengths and weaknesses of applying symbolic regression in genetic programming are evaluated against competitive techniques. In addition to software fault prediction this thesis also consolidates available research into predictive modeling of other attributes by applying symbolic regression in genetic programming, thus presenting a broader perspective. As an extension to the application of search-based techniques within software verification and validation this thesis further investigates the extent of application of search-based techniques for testing non-functional system properties. Based on the research findings in this thesis it can be concluded that applying symbolic regression in genetic programming may be a viable technique for software fault prediction. We additionally seek literature evidence where other search-based techniques are applied for testing of non-functional system properties, hence contributing towards the growing application of search-based techniques in diverse activities within software verification and validation.

  • 3. Afzal, Wasif
    Search-based prediction of software quality: Evaluations and comparisons2011Doctoral thesis, monograph (Other academic)
    Abstract [en]

    Software verification and validation (V&V) activities are critical for achieving software quality; however, these activities also constitute a large part of the costs when developing software. Therefore efficient and effective software V&V activities are both a priority and a necessity considering the pressure to decrease time-to-market and the intense competition faced by many, if not all, companies today. It is then perhaps not unexpected that decisions that affects software quality, e.g., how to allocate testing resources, develop testing schedules and to decide when to stop testing, needs to be as stable and accurate as possible. The objective of this thesis is to investigate how search-based techniques can support decision-making and help control variation in software V&V activities, thereby indirectly improving software quality. Several themes in providing this support are investigated: predicting reliability of future software versions based on fault history; fault prediction to improve test phase efficiency; assignment of resources to fixing faults; and distinguishing fault-prone software modules from non-faulty ones. A common element in these investigations is the use of search-based techniques, often also called metaheuristic techniques, for supporting the V&V decision-making processes. Search-based techniques are promising since, as many problems in real world, software V&V can be formulated as optimization problems where near optimal solutions are often good enough. Moreover, these techniques are general optimization solutions that can potentially be applied across a larger variety of decision-making situations than other existing alternatives. Apart from presenting the current state of the art, in the form of a systematic literature review, and doing comparative evaluations of a variety of metaheuristic techniques on large-scale projects (both industrial and open-source), this thesis also presents methodological investigations using search-based techniques that are relevant to the task of software quality measurement and prediction. The results of applying search-based techniques in large-scale projects, while investigating a variety of research themes, show that they consistently give competitive results in comparison with existing techniques. Based on the research findings, we conclude that search-based techniques are viable techniques to use in supporting the decision-making processes within software V&V activities. The accuracy and consistency of these techniques make them important tools when developing future decision support for effective management of software V&V activities.

  • 4.
    Afzal, Wasif
    Blekinge Institute of Technology.
    Using faults-slip-through metric as a predictor of fault-proneness2010In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC, 2010, p. 412-422Conference paper (Refereed)
    Abstract [en]

    Background: The majority of software faults are present in small number of modules, therefore accurate prediction of fault-prone modules helps improve software quality by focusing testing efforts on a subset of modules. Aims: This paper evaluates the use of the faults-slip-through (FST) metric as a potential predictor of fault-prone modules. Rather than predicting the fault-prone modules for the complete test phase, the prediction is done at the speci?c test levels of integration and system test. Method: We applied eight classi?cation techniques, to the task of identifying faultprone modules, representing a variety of approaches, including a standard statistical technique for classi?cation (logistic regression), tree-structured classi?ers (C4.5 and random forests), a Bayesian technique (Naïve Bayes), machine-learning techniques (support vector machines and back-propagation arti?cial neural networks) and search-based techniques (genetic programming and arti?cial immune recognition systems) on FST data collected from two large industrial projects from the telecommunication domain. Results: Using area under the receiver operating characteristic (ROC) curve and the location of (PF, PD) pairs in the ROC space, the faults-slip-through metric showed impressive results with the majority of the techniques for predicting fault-prone modules at both integration and system test levels. There were, however, no statistically signi?cant differences between the performance of different techniques based on AUC, even though certain techniques were more consistent in the classi?cation performance at the two test levels. Conclusions: We can conclude that the faults-slip-through metric is a potentially strong predictor of fault-proneness at integration and system test levels. The faults-slip-through measurements interact in ways that is conveniently accounted for by majority of the data mining techniques.

  • 5.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Bahria University, Islamabad, Pakistan.
    Alone, Snehal
    Chalmers University of Technology, Sweden.
    Glocksien, Kerstin
    Chalmers University of Technology, Sweden.
    Torkar, Richard
    Chalmers University of Technology, Sweden.
    Software Test Process Improvement Approaches: A Systematic Literature Review and an Industrial Case Study2016In: Journal of Systems and Software JSS, ISSN 0164-1212, Vol. 111, p. 1-33Article in journal (Refereed)
    Abstract [en]

    Software test process improvement (STPI) approaches are frameworks that guide software development organizations to improve their software testing process. We have identified existing STPI approaches and their characteristics (such as completeness of development, availability of information and assessment instruments, and domain limitations of the approaches) using a systematic literature review (SLR). Furthermore, two selected approaches (TPI NEXT and TMMi) are evaluated with respect to their content and assessment results in industry. As a result of this study, we have identified 18 STPI approaches and their characteristics. A detailed comparison of the content of TPI NEXT and TMMi is done. We found that many of the STPI approaches do not provide sufficient information or the approaches do not include assessment instruments. This makes it difficult to apply many approaches in industry. Greater similarities were found between TPI NEXT and TMMi and fewer differences. We conclude that numerous STPI approaches are available but not all are generally applicable for industry. One major difference between available approaches is their model representation. Even though the applied approaches generally show strong similarities, differences in the assessment results arise due to their different model representations.

  • 6.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bruneliere, H.
    IMT Atlantique – LS2N (CNRS) – ARMINES, France.
    Di Ruscio, D.
    Università degli Studi dell'Aquila - DISIM | Center of Excellence DEWS, Italy.
    Sadovykh, A.
    Softeam, France.
    Mazzini, S.
    Intecs, Italy.
    Cariou, E.
    Université de Pau et des Pays de l'Adour, LIUPPA, France.
    Truscan, D.
    Åbo Akademi University, Finland.
    Cabot, J.
    ICREA, Spain.
    Gómez, A.
    Internet Interdisciplinary Institute (IN3), Universitat Oberta de Catalunya (UOC), Spain.
    Gorroñogoitia, J.
    ATOS, Spain.
    Pomante, L.
    Università degli Studi dell'Aquila - DISIM | Center of Excellence DEWS, Italy.
    Smrz, P.
    Brno University of Technology, Czech Republic.
    The MegaM@Rt2 ECSEL project: MegaModelling at Runtime – Scalable model-based framework for continuous development and runtime validation of complex systems2018In: Microprocessors and microsystems, ISSN 0141-9331, E-ISSN 1872-9436, Vol. 61, p. 86-95Article in journal (Refereed)
    Abstract [en]

    A major challenge for the European electronic industry is to enhance productivity by ensuring quality of development, integration and maintenance while reducing the associated costs. Model-Driven Engineering (MDE) principles and techniques have already shown promising capabilities, but they still need to scale up to support real-world scenarios implied by the full deployment and use of complex electronic components and systems. Moreover, maintaining efficient traceability, integration, and communication between two fundamental system life cycle phases (design time and runtime) is another challenge requiring the scalability of MDE. This paper presents an overview of the ECSEL 1 project entitled “MegaModelling at runtime – Scalable model-based framework for continuous development and runtime validation of complex systems” (MegaM@Rt2), whose aim is to address the above mentioned challenges facing MDE. Driven by both large and small industrial enterprises, with the support of research partners and technology providers, MegaM@Rt2 aims to deliver a framework of tools and methods for: 1) system engineering/design and continuous development, 2) related runtime analysis and 3) global models and traceability management. Diverse industrial use cases (covering strategic domains such as aeronautics, railway, construction and telecommunications) will integrate and demonstrate the validity of the MegaM@Rt2 solution. This paper provides an overview of the MegaM@Rt2 project with respect to its approach, mission, objectives as well as to its implementation details. It further introduces the consortium as well as describes the work packages and few already produced deliverables.

  • 7.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bruneliere, Hugo
    AtlanMod Team, Inria, France.
    Di Ruscio, Davide
    Univ. of L'Aquila, L'Aquila, Italy.
    Sadovykh, Andrey
    Softeam, France.
    Mazzini, Silvia
    Intecs, Italy.
    Cariou, Eric
    Univ. de Pau et des Pays de l'Adour, Pau, France.
    Truscan, Dragos
    Åbo Akademi Univ., Turku, Finland.
    Cabot, Jordi
    Jordi Cabot ICREA, Barcelona, Spain.
    Field, Daniel
    ATOS, Madrid, Spain.
    Pomante, Luigi
    Univ. of L'Aquila, L'Aquila, Italy.
    Smrz, Pavel
    Brno Univ. of Technol., Brno, Czech Republic.
    The MegaM@Rt2 ECSEL Project: MegaModelling at Runtime — Scalable Model-Based Framework for Continuous Development and Runtime Validation of Complex Systems2017In: The 2017 Euromicro Conference on Digital System Design DSD'17, 2017Conference paper (Refereed)
    Abstract [en]

    A major challenge for the European electronic industry is to enhance productivity while reducing costs and ensuring quality in development, integration and maintenance. Model-Driven Engineering (MDE) principles and techniques have already shown promising capabilities but still need to scale to support real-world scenarios implied by the full deployment and use of complex electronic components and systems. Moreover, maintaining efficient traceability, integration and communication between two fundamental system life-time phases (design time and runtime) is another challenge facing scalability of MDE. This paper presents an overview of the ECSEL project entitled "MegaModelling at runtime -- Scalable model-based framework for continuous development and runtime validation of complex systems" (MegaM@Rt2), whose aim is to address the above mentioned challenges facing MDE. Driven by both large and small industrial enterprises, with the support of research partners and technology providers, MegaM@Rt2 aims to deliver a framework of tools and methods for: 1) system engineering/design & continuous development, 2) related runtime analysis and 3) global model & traceability management, respectively. The diverse industrial use cases (covering domains such as aeronautics, railway, construction and telecommunications) will integrate and apply such a framework that shall demonstrate the validation of the MegaM@Rt2 solution.

  • 8.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Caporuscio, M.
    Linnaeus University, Sweden.
    Conboy, H.
    University of Massachusetts Amherst, MA, United States.
    Di Marco, A.
    University of l'Aquila, Italy.
    Duchien, D. L.
    University of Lille, France.
    Pérez, D.
    University of British Columbia, Canada.
    Seceleanu, C.
    Kyushu University, Japan.
    Shahbazian, A.
    University of California, Berkeley, CA, United States.
    Spalazzese, R.
    Microsoft, WA, United States.
    Tivoli, M.
    Florida State University, FL, United States.
    Vasilescu, B.
    University College Dublin and Lero, Ireland.
    Washizaki, H
    Mälardalen University.
    Weyns, D.
    University of Southern California, CA, United States.
    Pasquale, L.
    Malmö University, Sweden.
    Nistor, A.
    Malmö University, Sweden.
    Muşlu, K.
    Waseda University, Japan.
    Kamei, Y.
    Waseda University, Japan.
    Hanam, Q.
    Carnegie Mellon University, PA, United States.
    Ying, A. T. T.
    Katholieke Universiteit Leuven, Belgium.
    Program committee for icse 2018 posters track2018In: Proceedings / International Conference of Software Engineering, ISSN 0270-5257, E-ISSN 1558-1225, Vol. Part F137351Article in journal (Refereed)
  • 9.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ghazi, Nauman
    Blekinge Institute of Technolog.
    Itkonen, Juha
    Aalto University, Espoo, Finland.
    Torkar, Richard
    Chalmers University of Technology.
    Andrews, Anneliese
    University of Denver, USA.
    Bhatti, Khurram
    Blekinge Institute of Technolog.
    An experiment on the effectiveness and efficiency of exploratory testing2015In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 20, no 3, p. 844-878Article in journal (Refereed)
    Abstract [en]

    The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test case based testing, TCT). We performed four controlled experiments where a total of 24 practitioners and 46 students performed manual functional testing using ET and TCT. We measured the number of identified defects in the 90-minute testing sessions, the detection difficulty, severity and types of the detected defects, and the number of false defect reports. The results show that ET found a significantly greater number of defects. ET also found significantly more defects of varying levels of difficulty, types and severity levels. However, the two testing approaches did not differ significantly in terms of the number of false defect reports submitted. We conclude that ET was more efficient than TCT in our experiment. ET was also more effective than TCT when detection difficulty, type of defects and severity levels are considered. The two approaches are comparable when it comes to the number of false defect reports submitted.

  • 10.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering. Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    A comparative evaluation of using genetic programming for predicting fault count data2008In: Proceedings - The 3rd International Conference on Software Engineering Advances, ICSEA 2008, Includes ENTISY 2008: International Workshop on Enterprise Information Systems, 2008, p. 407-414Conference paper (Refereed)
    Abstract [en]

    There have been a number of software reliability growth models (SRGMs) proposed in literature. Due to several reasons, such as violation of models’ assumptions and complexity of models, the practitioners face difficulties in knowing which models to apply in practice. This paper presents a comparative evaluation of traditional models and use of genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The motivation of using a GP approach is its ability to evolve a model based entirely on prior data without the need of making underlying assumptions. The results show the strengths of using GP for predicting fault count data.

  • 11.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Incorporating metrics in an organizational test strategy2008In: International Conference on Software Testing, Verification and Validation: Proceedings of the International Software Testing Standard Workshop, Collocated with 1st International Conference on Software Testing, Verification and Validation, 2008, p. 304-315Conference paper (Refereed)
    Abstract [en]

    An organizational level test strategy needs to incorporate metrics to make the testing activities visible and available to process improvements. The majority of testing measurements that are done are based on faults found in the test execution phase. In contrast, this paper investigates metrics to support software test planning and test design processes. We have assembled metrics in these two process types to support management in carrying out evidence-based test process improvements and to incorporate suitable metrics as part of an organization level test strategy. The study is composed of two steps. The first step creates a relevant context by analyzing key phases in the software testing lifecycle, while the second step identifies the attributes of software test planning and test design processes along with metric(s) support for each of the identified attributes.

  • 12.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Lessons from applying experimentation in software engineering predictive modeling2008In: Proceedings of The 2nd International workshop on Software Productivity Analysis and Cost Estimation (SPACE'08), Collocated with 15th Asia-Pacific Software Engineering Conference, Beijing, China, 2008Conference paper (Refereed)
    Abstract [en]

    Within software engineering prediction systems, experiments are undertaken primarily to investigate relationships and to measure/compare models’ accuracy. This paper discusses our experience and presents useful lessons/guidelines in experimenting with software engineering prediction systems. For this purpose, we use a typical software engineering experimentation process as a baseline. We found that the typical experimentation process in software engineering is supportive in developing prediction systems and have highlighted issues more central to the domain of software engineering prediction systems.

  • 13.
    Afzal, Wasif
    et al.
    Blekinge Inst Technol.
    Torkar, Richard
    Blekinge Inst Technol.
    On the application of genetic programming for software engineering predictive modeling: A systematic review2011In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 38, no 9, p. 11984-11997Article in journal (Refereed)
    Abstract [en]

    The objective of this paper is to investigate the evidence for symbolic regression using genetic programming (GP) being an effective method for prediction and estimation in software engineering, when compared with regression/machine learning models and other comparison groups (including comparisons 20 with different improvements over the standard GP algorithm). We performed a systematic review of literature that compared genetic programming models with comparative techniques based on different 22 independent project variables. A total of 23 primary studies were obtained after searching different information sources in the time span 1995–2008. The results of the review show that symbolic regression using genetic programming has been applied in three domains within software engineering predictive modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estimation (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight primary studies). While there is evidence in support of using genetic programming for software quality classification, software fault prediction and software reliability growth modeling; the results are inconclusive for software cost/effort/size estimation.

  • 14.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology, Sweden.
    Torkar, Richard
    Blekinge Institute of Technology, Sweden.
    Suitability of genetic programming for software reliability growth modelling2008In: Proceedings - International Symposium on Computer Science and Its Applications, CSA 2008, 2008, p. 114-117Conference paper (Refereed)
    Abstract [en]

    Genetic programming (GP) has been found to be effective in finding a model that fits the given data points without making any assumptions about the model structure. This makes GP a reasonable choice for software reliability growth modeling. This paper discusses the suitability of using GP for software reliability growth modeling and highlights the mechanisms that enable GP to progressively search for fitter solutions.

  • 15.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Bahria University, Islamabad, Pakistan .
    Torkar, Richard
    Blekinge Institute of Technology, Karlskrona, Sweden; Chalmers University of Technology, Sweden.
    Towards benchmarking feature subset selection methods for software fault prediction2016In: Computational Intelligence and Quantitative Software Engineering / [ed] Witold Pedrycz, Giancarlo Succi and Alberto Sillitti, Springer-Verlag , 2016, p. 33-58Chapter in book (Other academic)
    Abstract [en]

    Despite the general acceptance that software engineering datasets often contain noisy, irrele- vant or redundant variables, very few benchmark studies of feature subset selection (FSS) methods on real-life data from software projects have been conducted. This paper provides an empirical comparison of state-of-the-art FSS methods: information gain attribute ranking (IG); Relief (RLF); principal com- ponent analysis (PCA); correlation-based feature selection (CFS); consistency-based subset evaluation (CNS); wrapper subset evaluation (WRP); and an evolutionary computation method, genetic program- ming (GP), on five fault prediction datasets from the PROMISE data repository. For all the datasets, the area under the receiver operating characteristic curve—the AUC value averaged over 10-fold cross- validation runs—was calculated for each FSS method-dataset combination before and after FSS. Two diverse learning algorithms, C4.5 and na ??ve Bayes (NB) are used to test the attribute sets given by each FSS method. The results show that although there are no statistically significant differences between the AUC values for the different FSS methods for both C4.5 and NB, a smaller set of FSS methods (IG, RLF, GP) consistently select fewer attributes without degrading classification accuracy. We conclude that in general, FSS is beneficial as it helps improve classification accuracy of NB and C4.5. There is no single best FSS method for all datasets but IG, RLF and GP consistently select fewer attributes without degrading classification accuracy within statistically significant boundaries.

  • 16.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    A systematic mapping study on non-functional search-based software testing2008In: 20th International Conference on Software Engineering and Knowledge Engineering, 2008Conference paper (Refereed)
    Abstract [en]

    Automated software test generation has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional), greybox (combination of structural and functional) and nonfunctional testing. In this paper, we undertake a systematic mapping study to present a broad review of primary studies on the application of search-based optimization techniques to non-functional testing. The motivation is to identify the evidence available on the topic and to identify gaps in the application of search-based optimization techniques to different types of non-functional testing. The study is based on a comprehensive set of 35 papers obtained after using a multi-stage selection criteria and are published in workshops, conferences and journals in the time span 1996–2007. We conclude that the search-based software testing community needs to do more and broader studies on nonfunctional search-based software testing (NFSBST) and the results from our systematic map can help direct such efforts.

  • 17.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    A systematic review of search-based testing for non-functional system properties2009In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 51, no 6, p. 957-976Article, review/survey (Refereed)
    Abstract [en]

    Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996–2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.

  • 18.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technolog.
    Torkar, Richard
    Blekinge Institute of Technolog.
    Feldt, Robert
    Blekinge Institute of Technology.
    Prediction of fault count data using genetic programming2008In: IEEE INMIC 2008: 12th IEEE International Multitopic Conference - Conference Proceedings, 2008, p. 349-356Conference paper (Refereed)
    Abstract [en]

    Software reliability growth modeling helps in deciding project release time and managing project resources. A large number of such models have been presented in the past. Due to the existence of many models, the models’ inherent complexity, and their accompanying assumptions; the selection of suitable models becomes a challenging task. This paper presents empirical results of using genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The goodness of ?t (adaptability) and predictive accuracy of the evolved model is measured using ?ve different measures in an attempt to present a fair evaluation. The results show that the GP evolved model has statistically signi?cant goodness of ?t and predictive accuracy

  • 19.
    Afzal, Wasif
    et al.
    Bahria Univ, Islamabad, Pakistan.
    Torkar, Richard
    Chalmers University of Technology and the University of Gothenburg.
    Feldt, Robert
    Blekinge Institute of Technology.
    Resampling methods in software quality classification2012In: International journal of software engineering and knowledge engineering, ISSN 0218-1940, Vol. 22, no 2, p. 203-223Article in journal (Refereed)
    Abstract [en]

    In the presence of a number of algorithms for classification and prediction in software engineering, there is a need to have a systematic way of assessing their performances. The performance assessment is typically done by some form of partitioning or resampling of the original data to alleviate biased estimation. For predictive and classification studies in software engineering, there is a lack of a definitive advice on the most appropriate resampling method to use. This is seen as one of the contributing factors for not being able to draw general conclusions on what modeling technique or set of predictor variables are the most appropriate. Furthermore, the use of a variety of resampling methods make it impossible to perform any formal meta analysis of the primary study results. Therefore, it is desirable to examine the influence of various resampling methods and to quantify possible differences. Objective and method: This study empirically compares five common resampling methods (hold-out validation, repeated random sub-sampling, 10-fold cross-validation, leave-one-out cross-validation and nonparametric bootstrapping) using 8 publicly available data sets with genetic programming (GP) and multiple linear regression (MLR) as software quality classification approaches. Location of (PF, PD) pairs in the ROC (receiver operating characteristics) space and area under an ROC curve (AUC) are used as accuracy indicators. Results: The results show that in terms of the location of (PF, PD) pairs in the ROC space, bootstrapping results are in the preferred region for 3 of the 8 data sets for GP and for 4 of the 8 data sets for MLR. Based on the AUC measure, there are no significant differences between the different resampling methods using GP and MLR. Conclusion: There can be certain data set properties responsible for insignificant differences between the resampling methods based on AUC. These include imbalanced data sets, insignificant predictor variables and high-dimensional data sets. With the current selection of data sets and classification techniques, bootstrapping is a preferred method based on the location of (PF, PD) pair data in the ROC space. Hold-out validation is not a good choice for comparatively smaller data sets, where leave-one-out cross-validation (LOOCV) performs better. For comparatively larger data sets, 10-fold cross-validation performs better than LOOCV.

  • 20.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    Search-based prediction of fault count data2009In: Proceedings - 1st International Symposium on Search Based Software Engineering, SSBSE 2009, 2009, p. 35-38Conference paper (Refereed)
    Abstract [en]

    Symbolic regression, an application domain of genetic programming (GP), aims to find a function whose output has some desired property, like matching target values of a particular data set. While typical regression involves finding the coefficients of a pre-defined function,symbolic regression finds a general function, with coefficients,fitting the given set of data points. The conceptsof symbolic regression using genetic programming can be used to evolve a model for fault countpredictions.Such a model has the advantages that the evolution is not dependent on a particular structure of the model and is also independent of any assumptions, which are common in traditional time-domain parametric software reliability growth models. This research aims at applying experiments targeting fault predictionsusing genetic programming and comparing the results with traditional approaches to compare efficiency gains

  • 21.
    Afzal, Wasif
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    Gorschek, Tony
    Blekinge Institute of Technology.
    Genetic programming for cross-release fault count predictions in large and complex software projects2009In: Evolutionary Computation and Optimization Algorithms in Software Engineering / [ed] Monica Chis, IGI Global, 2009Chapter in book (Other academic)
    Abstract [en]

    Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multi release open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.

  • 22.
    Afzal, Wasif
    et al.
    Bahria Univ, Pakistan.
    Torkar, Richard
    Chalmers University of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    Gorschek, Tony
    Blekinge Institute of Technology.
    Prediction of faults-slip-through in large software projects: An empirical evaluation2013In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 22, no 1, p. 51-86Article in journal (Refereed)
    Abstract [en]

    A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization-based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.

  • 23.
    Afzal, Wasif
    et al.
    Mälardalen University, School of Innovation, Design and Engineering. Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Feldt, Robert
    Blekinge Institute of Technology.
    Wikstrand, Greger
    KnowIT YAHM Sweden AB.
    Search-based prediction of fault-slip-through in large software projects2010In: Proceedings - 2nd International Symposium on Search Based Software Engineering, SSBSE 2010, 2010, p. 79-88Conference paper (Refereed)
    Abstract [en]

    A large percentage of the cost of rework can be avoided by ?nding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of ?ve different techniques, namely particle swarm optimization based arti?cial neural networks (PSO-ANN), arti?cial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards ?nding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of ?t and absolute relative error measures. They indicate that the four search-based techniques (PSOANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.

  • 24.
    Ahmed, B. S.
    et al.
    Istituto Dalle Molle di Studi Sull'Intelligenza Artificiale (IDSIA), Manno-Lugano, Switzerland.
    Sahib, M. A.
    Software and Informatics Engineering Department, Engineering College, Salahaddin University - Erbil, Iraq.
    Gambardella, L. M.
    Istituto Dalle Molle di Studi Sull'Intelligenza Artificiale (IDSIA), Manno-Lugano, Switzerland.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Zamli, K. Z.
    IBM Centre of Excellence, Faculty of Computer Systems and Software Engineering, Universiti Malaysia Pahang Lebuhraya Tun Razak, Kuantan, Pahang Darul Makmur, Malaysia.
    Optimum design of PIλDμ controller for an automatic voltage regulator system using combinatorial test design2016In: PLoS ONE, ISSN 1932-6203, E-ISSN 1932-6203, Vol. 11, no 11, article id e0166150Article in journal (Refereed)
    Abstract [en]

    Combinatorial test design is a plan of test that aims to reduce the amount of test cases systematically by choosing a subset of the test cases based on the combination of input variables. The subset covers all possible combinations of a given strength and hence tries to match the effectiveness of the exhaustive set. This mechanism of reduction has been used successfully in software testing research with t-way testing (where t indicates the interaction strength of combinations). Potentially, other systems may exhibit many similarities with this approach. Hence, it could form an emerging application in different areas of research due to its usefulness. To this end, more recently it has been applied in a few research areas successfully. In this paper, we explore the applicability of combinatorial test design technique for Fractional Order (FO), Proportional-Integral-Derivative (PID) parameter design controller, named as FOPID, for an automatic voltage regulator (AVR) system. Throughout the paper, we justify this new application theoretically and practically through simulations. In addition, we report on first experiments indicating its practical use in this field. We design different algorithms and adapted other strategies to cover all the combinations with an optimum and effective test set. Our findings indicate that combinatorial test design can find the combinations that lead to optimum design. Besides this, we also found that by increasing the strength of combination, we can approach to the optimum design in a way that with only 4-way combinatorial set, we can get the effectiveness of an exhaustive test set. This significantly reduced the number of tests needed and thus leads to an approach that optimizes design of parameters quickly. © 2016 Ahmed et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

  • 25.
    Ahmed, Bestoun
    et al.
    Istituto Dalle Molle di Studi sullIntelligenza Artificiale (IDSIA), Switzerland.
    Gambardella, Luca
    Istituto Dalle Molle di Studi sullIntelligenza Artificiale (IDSIA), Switzerland.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Zamli, Kamal
    University Malaysia Pahang, Gambang, Malaysia.
    Handling Constraints in Combinatorial Interaction Testing in the Presence of Multi Objective Particle Swarm and Multithreading2017In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 86, no 01, p. 20-36Article in journal (Refereed)
    Abstract [en]

    Combinatorial strategies have received a lot of attention lately as a result of their diverse applications in areas of research, particularly in software engineering. In its simple form, a combinatorial strategy can reduce several input parameters (configurations) of a system into a small set of these parameters based on their interaction (combination). However, in practice, the input configurations of software systems are subjected to constraints, especially highly configurable systems. To implement this feature within a strategy, many difficulties arise for construction. While there are many combinatorial interaction testing strategies nowadays, few of them support constraints. This paper presents a new strategy, called Octopus to construct a combinatorial interaction test suites with the presence of constraints. The design and algorithms are provided in the paper in detail. The strategy is inspired by the behaviour of octopus to search for the optimal solution using multi-threading mechanism. To overcome the multi judgement criteria for an optimal solution, the multi-objective particle swarm optimisation is used. The strategy and its algorithms are evaluated extensively using different benchmarks and comparisons. The evaluation results showed the efficiency of each algorithm in the strategy. The benchmarking results also showed that Octopus can generate test suites efficiently as compared to state-of-the-art strategies.

  • 26.
    Ahmed, Bestoun
    et al.
    Czech Technical University, Czech Republic.
    Zamli, Kamal
    University Malaysia Pahang, Gambang, Malaysia..
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bures, Miroslav
    Czech Technical University, Czech Republic.
    Constrained Interaction Testing: A Systematic Literature Study2017In: IEEE Access, E-ISSN 2169-3536, IEEE Access, ISSN 2169-3536, Vol. PP, no 99Article, book review (Other academic)
    Abstract [en]

    Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data is extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation & application and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this study also discusses the applications of constrained interaction testing in several domains such as software product lines, fault detection & characterization, test selection, security and GUI testing. The study ends with a discussion of limitations, challenges and future work in the area.

  • 27.
    Bashir, Shariq
    et al.
    Mohammad Ali Jinnah University, Islamabad, Pakistan.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Baig, Rauf
    Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia.
    Opinion-based entity ranking using learning to rank2016In: Applied Soft Computing, ISSN 1568-4946, E-ISSN 1872-9681, Vol. 38, no 1, p. 151-163Article in journal (Refereed)
    Abstract [en]

    As social media and e-commerce on the Internet continue to grow, opinions have become one of the most important sources of information for users to base their future decisions on. Unfortunately, the large quantities of opinions make it difficult for an individual to comprehend and evaluate them all in a reasonable amount of time. The users have to read a large number of opinions of different entities before making any decision. Recently a new retrieval task in information retrieval known as Opinion-Based Entity Ranking (OpER) has emerged. OpER directly ranks relevantentities based on how well opinions on them are matched with a user's preferences that are given in the form of queries. With such a capability, users do not need to read a large number of opinions available for the entities. Previous research on OpER does not take into account the importance and subjectivity of query keywords in individual opinions of an entity. Entity relevance scores are computed primarily on the basis of occurrences of query keywords match, by assuming all opinions of an entity as a single field of text. Intuitively, entities that have positive judgments and strong relevance with query keywords should be ranked higher than those entities that have poor relevance and negative judgments. This paper outlines several ranking features and develops an intuitive framework for OpER in which entities are ranked according to how well individual opinions of entities are matched with the user's query keywords. As a useful ranking model may be constructed from many rankingfeatures, we apply learning to rank approach based on genetic programming (GP) to combine features in order to develop an effective retrieval model for OpER task. The proposed approach is evaluated on two collections and is found to be significantly more effective than the standard OpER approach.

  • 28.
    Betz, Stefanie
    et al.
    Blekinge Institute of Technology.
    Smite, Darja
    Blekinge Institute of Technology.
    Fricker, Samuel
    Blekinge Institute of Technology.
    Moss, Andrew
    Blekinge Institute of Technology.
    Afzal, Wasif
    Bahria University, Pakistan.
    Svahnberg, Mikael
    Blekinge Institute of Technology.
    Wohlin, Claes
    Blekinge Institute of Technology.
    Börstler, Jürgen
    Blekinge Institute of Technology.
    Gorschek, Tony
    Blekinge Institute of Technology.
    An evolutionary perspective on socio-technical congruence: The rubber band effect2013In: International Symposium on Empirical Software Engineering and Measurement: Proceedings of the 3rd International Workshop on Replication in Empirical Software Engineering Research (RESER'13), Collocated with 7th International Symposium on Empirical Software Engineering and Measurement (ESEM'13), Baltimore, United States, 2013Conference paper (Refereed)
    Abstract [en]

    Conway’s law assumes a strong association between the system’s architecture and the organization’s communication structure that designs it. In the light of contemporary software development, when many companies rely on geographically distributed teams, which often turn out to be temporarily composed and thus having an often changing communication structure, the importance of Conway’s law and its inspired work grows. In this paper, we examine empirical research related to Conway’s law and its application for cross-site coordination. Based on the results obtained we conjecture that changes in the communication structure alone sooner or later trigger changes in the design structure of the software products to return the sociotechnical system into the state of congruence. This is further used to formulate a concept of a rubber band effect and propose a replication study that goes beyond the original idea of Conway’s law by investigating the evolution of sociotechnical congruence over time.

  • 29.
    Brahneborg, D.
    et al.
    Infoflex Connect AB, Stockholm, Sweden.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Björkman, Mats
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Towards a more reliable store-and-forward protocol for mobile text messages2018In: Proceedings of the Annual ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery , 2018, p. 13-20Conference paper (Refereed)
    Abstract [en]

    Businesses often use mobile text messages (SMS) as a cost effective and universal way of communicating concise information to their customers. Today, these messages are usually sent via SMS brokers, which forward them further to the next stakeholder, typically the various mobile operators, and then the messages eventually reach the intended recipients. Infoflex Connect AB delivers an SMS gateway application to the brokers with the main responsibility of reliable message delivery within set quality thresholds. However, the protocols used for SMS communication are not designed for reliability and thus messages may be lost. In this position paper we deduce requirements for a new protocol for routing messages through the SMS gateway application running at a set of broker nodes, in order to increase the reliability. The requirements cover important topics for the required communication protocol such as event ordering, message handling and system membership. The specification of such requirements sets the foundation for the forthcoming design and implementation of such a protocol and its evaluation.

  • 30.
    Brahneborg, Daniel
    et al.
    Infoflex Connect AB, Stockholm, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Black-Box Approach to Latency and Throughput Analysis2017In: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017, 2017, p. 603-604, article id 8004393Conference paper (Refereed)
    Abstract [en]

    To enable fast and reliable delivery of mobile text messages (SMS), special bidirectional protocols are often used. Measuring the achieved throughput and involved latency is however non-trivial, due to the complexity of these protocols. Modifying an existing system would incur too much of a risk, so instead a new tool was created to analyse the log files containing information about this traffic in a black-box fashion. When the produced raw data was converted into graphs, they gave new insights into the behaviour of both the protocols and the remote systems involved.

  • 31.
    Brahneborg, Daniel
    et al.
    Infoflex Connect AB, Stockholm, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Pragmatic Perspective on Regression Testing Challenges2017In: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017, Prague, Czech Republic, 2017, p. 618-619, article id 8004401Conference paper (Refereed)
    Abstract [en]

    Regression testing research has received significant focus during the past decades, acknowledging the benefits it can provide to organisations in terms of reduced development and maintenance costs, as well as sustained end-user satisfaction. There are several challenges left to overcome before the industry can fully take advantage of the available research results in this area. To get a better overview of how current regression testing research fits in with today’s industrial practices, we read a selection of papers in the field and based on our experience, critically examined their content. As a result, we present and discuss a taxonomy of regression testing challenges, from the perspectives of both methods and organisations, that we believe will foster the industrial uptake of regression testing.

  • 32.
    Doganay, Kivanc
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, S.
    Ericsson AB, Kista, Sweden .
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Search-based testing for embedded telecom software with complex input structures2014In: Testing Software and Systems: 26th IFIP WG 6.1 International Conference, ICTSS 2014, Madrid, Spain, September 23-25, 2014. Proceedings, 2014, p. 205-210Conference paper (Refereed)
    Abstract [en]

    In this paper, we discuss the application of search-based software testing techniques for unit level testing of a real-world telecommunication middleware at Ericsson. Our current implementation analyzes the existing test cases to handle non-trivial variables such as uninitialized pointers, and to discover any setup code that needs to run before the actual test case, such as setting global system parameters. Hill climbing (HC) and (1+1) evolutionary algorithm (EA) metaheuristic search algorithms are used to generate input data for branch coverage. We compare HC, (1+1)EA, and random search with respect to effectiveness, measured as branch coverage, and efficiency, measured as number of executions needed. Difficulties arising from the specialized execution environment and the adaptations for handling these problems are also discussed.

  • 33.
    Doganay, Kivanc
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. SICS Swedish ICT AB, Kista, Sweden..
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.;Karlstad Univ, Karlstad, Sweden..
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. SICS Swedish ICT AB, Kista, Sweden.
    Search-Based Testing for Embedded Telecom Software with Complex Input Structures2014In: TESTING SOFTWARE AND SYSTEMS (ICTSS 2014) / [ed] Merayo, MG DeOca, EM, SPRINGER-VERLAG BERLIN , 2014, p. 205-210Conference paper (Refereed)
    Abstract [en]

    In this paper, we discuss the application of search-based software testing techniques for unit level testing of a real-world telecommunication middleware at Ericsson. Our current implementation analyzes the existing test cases to handle non-trivial variables such as uninitialized pointers, and to discover any setup code that needs to run before the actual test case, such as setting global system parameters. Hill climbing (HC) and (1+1) evolutionary algorithm (EA) metaheuristic search algorithms are used to generate input data for branch coverage. We compare HC, (1+1) EA, and random search with respect to effectiveness, measured as branch coverage, and efficiency, measured as number of executions needed. Difficulties arising from the specialized execution environment and the adaptations for handling these problems are also discussed.

  • 34.
    Doganay, Kivanc
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Search-based Testing for Embedded Telecommunication Software with Complex Input Structures: An Industrial Case Study2014Report (Other academic)
    Abstract [en]

    In this paper, we discuss the application of search-based software test-ing techniques for unit level testing of a real-world telecommunication middleware at Ericsson. Input data for the system under test consists of nested data structures, and includes non-trivial variables such as unini-tialized pointers. Our current implementation analyzes the existing test cases to discover how to handle pointers, set global system parameters, and any other setup code that needs to run before the actual test case. Hill climbing (HC) and (1+1) evolutionary algorithm (EA) metaheuristic search algorithms are used to generate input data for branch coverage. We compare HC, (1+1)EA, and random search as a baseline of performance with respect to e˙ectiveness, measured as branch coverage, and eÿciency, measured as number of executions needed. Diÿculties arising from the specialized execution environment and the adaptations for handling these problems are also discussed.

  • 35.
    Feldt, Robert
    et al.
    Blekinge Institute of Technology.
    Torkar, Richard
    Blekinge Institute of Technology.
    Gorschek, Tony
    Blekinge Institute of Technology.
    Afzal, Wasif
    Blekinge Institute of Technology.
    Searching for cognitively diverse tests: Towards universal test diversity metrics2008In: International Conference on Software Testing, Verification and Validation: Proceedings of the 1st International Workshop on Search-based Software Testing (SBST'08), Collocated with 1st International Conference on Software Testing, Verification and Validation, 2008, p. 178-186Conference paper (Refereed)
    Abstract [en]

    Search-based software testing (SBST) has shown a potential to decrease cost and increase quality of testing related software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness functions that measure test fitness in relation to existing or previously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at variation points in the model. We then describe how to apply a practically useful approximation to the theoretically optimal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measurement scenarios. We present initial results from an experiment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation in an object-oriented, dynamic programming language.

  • 36.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Exploring Test Overlap in System Integration: An Industrial Case Study2016In: 42nd Euromicro Conference series on Software Engineering and Advanced Applications SEAA 2016, 2016, p. 303-312Conference paper (Refereed)
    Abstract [en]

    Tougher safety regulations, global competition and ever increasing complexity of embedded software puts extensive pressure on the effectiveness of the software testing procedures. Previous studies have found that there exist overlaps (i.e., multiple instances of highly similar test cases) and even redundancies in the software testing process. Such overlap has been found between versions, variants and integration levels, but primarily at unit test level. Given large embedded systems involving many subsystems, does overlap exist within the system integration testing as well? In this paper, we present an industrial case study, aiming to a) evaluate if there exist test overlaps within the given context, b) if so, investigate how these overlaps are distributed, and c) find ways of reducing test effort by investigating how the knowledge of overlaps and their distribution may be used for finding candidate test cases for automation, maintenance or even removal. We have studied manual test cases, written in natural language, at a large vehicular manufacturer in Sweden. In particular, we have collected and analyzed test cases from the system integration testing levels of four different projects of a vehicle control management system. Using a similarity function, we evaluate if any overlaps between test cases exist, and where. We found that overlaps do exist within the system integration level, particularly in the form of partial test step sequences. However, very few test cases overlapped in their entirety. Some candidates for test step automation and update propagation were identified, but none for easy removal.

  • 37.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Gustafsson, T.
    Scania CV AB, Sweden.
    Kobetski, A.
    RISE SICS AB, Sweden.
    From natural language requirements to passive test cases using guarded assertions2018In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 470-481Conference paper (Refereed)
    Abstract [en]

    In large-scale embedded system development, requirements are often expressed in natural language. Translating these requirements to executable test cases, while keeping the test cases and requirements aligned, is a challenging task. While such a transformation typically requires extensive domain knowledge, we show that a systematic process in combination with passive testing would facilitate the translation as well as linking the requirements to tests. Passive testing approaches observe the behavior of the system and test their correctness without interfering with the normal behavior. We use a specific approach to passive testing: guarded assertions (G/A). This paper presents a method for transforming system requirements expressed in natural language into G/As. We further present a proof of concept evaluation, performed at Bombardier Transportation Sweden AB, in which we show how the process would be used, together with practical advice of the reasoning behind the translation steps.

  • 38.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study2015In: The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, 2015, p. 317-324Conference paper (Refereed)
    Abstract [en]

    Vertical test reuse refers to the the reuse of test cases or other test artifacts over different integration levels in the software or system engineering process. Vertical test reuse has previously been proposed for reducing test effort and improving test effectiveness, particularly for embedded system development. The goal of this study is to provide an overview of the state of the art in the field of vertical test reuse for embedded system development. For this purpose, a systematic mapping study has been performed, identifying 11 papers on vertical test reuse for embedded systems. The primary result from the mapping is a classification of published work on vertical test reuse in the embedded system domain, covering motivations for reuse, reuse techniques, test levels and reusable test artifacts considered, and to what extent the effects of reuse have been evaluated.

  • 39.
    Garousi, Vahid
    et al.
    University of Luxembourg, Luxembourg.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Çağlar, Adem
    Havelsan, Turkey.
    İhsan Berk Işık, Berk Işık
    Havelsan, Turkey.
    Baydan, Berker
    Havelsan, Turkey.
    Çaylak, Seçkin
    Havelsan, Turkey.
    Boyraz, Ahmet Zeki
    Havelsan, Turkey.
    Yolaçan, Burak
    Havelsan, Turkey.
    Herkiloğlu, Kadir
    Comparing Automated Visual GUI Testing Tools: An Industrial Case Study2017In: The 8th ACM SIGSOFT International Workshop on Automated Software Testing A-TEST'17, 2017, p. 21-28Conference paper (Refereed)
    Abstract [en]

    Visual GUI testing (VGT) is a tool-driven technique, which uses image recognition for interaction and assertion of the behaviour of system under test. Motivated by a real industrial need, in the context of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, we systematically planned and applied a VGT project in this industrial context. The goal of the initial phase of the project was to empirically evaluate two well-known VGT tools (Sikuli and JAutomate) to help the company select the best tool for a given testing project. Our results show that both two tools suffer from similar test ‘Replay’ problems such as the inability to find smaller-sized images. The repeatability of test executions was better for JAutomate in case of one of the two software under test (SUT) while it was comparable for the other. In terms of test development effort, for both tools, there were high correlations with number of steps in test suites, however the effort is reduced if test code is reused. The study has already provided benefits to the test engineers and managers in the company by increasing the know-how in the company w.r.t. VGT, and by identifying the challenges and their workarounds in using the tools. The industrial case study in this paper intends to add to the body of evidence in VGT and help other researchers and practitioners.

  • 40.
    Hassan, Mohammad Mahdi
    et al.
    Karlstad University, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Blom, Martin
    Karlstad universitet, Sweden.
    Lindström, Birgitta
    Högskolan i Skövde, Sweden.
    Andler, Sten F.
    Högskolan i Skövde, Sweden.
    Eldh, Sigrid
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Testability and Software Robustness: A Systematic Literature Review2015In: The 41st Euromicro Conference on Software Engineering and Advanced Applications SEAA'15, Funchal, Madeira, Portugal, 2015, p. 341-348Conference paper (Refereed)
    Abstract [en]

    The concept of software testability has been researched in several different dimensions, however the relation of this important concept with other quality attributes is a grey area where existing evidence is scattered. The objective of this study is to present a state-of-the-art with respect to issues of importance concerning software testability and an important quality attribute: software robustness. The objective is achieved by conducting a systematic literature review (SLR) on the topic. Our results show that a variety of testability issues are in focus with observability and controllability issues being most researched. Fault tolerance, exception handling and handling external influence are prominent robustness issues in focus.

  • 41.
    Hassan, Mohammad Mahdi
    et al.
    Karlstad University, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Lindström, Birgitta
    University of Skövde, Sweden.
    Ali Shah, Syed Muhammad
    SICS, Sweden.
    Andler, Sten F.
    University of Skövde, Sweden.
    Blom, Martin
    Karlstad University, Sweden.
    Testability and software performance: A systematic mapping study2016In: SAC '16 Proceedings of the 31st Annual ACM Symposium on Applied Computing, 2016, p. 1566-1569Conference paper (Refereed)
    Abstract [en]

    In most of the research on software testability, functional correctness of the software has been the focus while the evidence regarding testability and non-functional properties such as performance is sporadic. The objective of this study is to present the current state-of-the-art related to issues of importance, types and domains of software under test, types of research, contribution types and design evaluation methods concerning testability and software performance. We find that observability, controllability and testing effort are the main testability issues while timeliness and response time (i.e., time constraints) are the main performance issues in focus. The primary studies in the area use diverse types of software under test within different domains, with realtime systems as being a dominant domain. The researchers have proposed many different methods in the area, however these methods lack implementation in practice.

  • 42.
    Irshad, M.
    et al.
    Blekinge Institute of Technology, Karlskrona, Sweden.
    Torkar, R.
    Chalmers University of Technology, Gothenburg, Sweden.
    Petersen, K.
    Blekinge Institute of Technology, Karlskrona, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Capturing cost avoidance through reuse: Systematic literature review and industrial evaluation2016In: ACM International Conference Proceeding Series, 2016, Vol. 01, article id Article number a35Conference paper (Refereed)
    Abstract [en]

    Background: Cost avoidance through reuse shows the benefits gained by the software organisations when reusing an artefact. Cost avoidance captures benefits that are not captured by cost savings e.g. spending that would have increased in the absence of the cost avoidance activity. This type of benefit can be combined with quality aspects of the product e.g. costs avoided because of defect prevention. Cost avoidance is a key driver for software reuse. Objectives: The main objectives of this study are: (1) To assess the status of capturing cost avoidance through reuse in the academia; (2) Based on the first objective, propose improvements in capturing of reuse cost avoidance, integrate these into an instrument, and evaluate the instrument in the software industry. Method: The study starts with a systematic literature review (SLR) on capturing of cost avoidance through reuse. Later, a solution is proposed and evaluated in the industry to address the shortcomings identified during the systematic literature review. Results: The results of a systematic literature review describe three previous studies on reuse cost avoidance and show that no solution, to capture reuse cost avoidance, was validated in industry. Afterwards, an instrument and a data collection form are proposed that can be used to capture the cost avoided by reusing any type of reuse artefact. The instrument and data collection form (describing guidelines) were demonstrated to a focus group, as part of static evaluation. Based on the feedback, the instrument was updated and evaluated in industry at 6 development sites, in 3 different countries, covering 24 projects in total. Conclusion: The proposed solution performed well in industrial evaluation. With this solution, practitioners were able to do calculations for reuse costs avoidance and use the results as decision support for identifying potential artefacts to reuse.

  • 43.
    Mehmood, M. A.
    et al.
    Balochistan University of IT, Eng. and Management Sciences and Shaheed Zulfikar Ali Bhutto Institute of Science and Technology, Pakistan.
    Khan, M. N. A.
    Shaheed Zulfikar Ali Bhutto Institute of Science and Technology, Pakistan.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Transforming context-aware application development model into a testing model2018In: Proceedings of the IEEE International Conference on Software Engineering and Service Sciences, ICSESS, IEEE Computer Society , 2018, p. 177-182Conference paper (Refereed)
    Abstract [en]

    Software testing aims at ensuring the quality of a software product. Context-aware applications are emerging applications that are capable to sense their environment and adapt to situational context to provide better user experience. Context-aware applications pose many challenges for software testing such as defining test adequacy criteria, generating test data, developing context-aware test cases etc. Test case generation process for context-aware applications can be automated using a model based testing technique. To attain this goal with maximum degree of automation, it is required to transform development model into a test model automatically. In this study, we propose a typecast of activity node of UML activity diagram, called Context-Aware Activity for modelling context-aware applications. We have also developed an approach for automatic transformation of the development model i.e., UML activity diagram with Context-aware Activity typecast into a testing model i.e. function nets. This testing model is used to automate test case generation and we have illustrated how to generate context-aware test cases using our proposed approach.

  • 44.
    Muellner, Nils
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Khan, S.
    Carl von Ossietzky Universität, Oldenburg, Germany .
    Rahman, M. H.
    Carl von Ossietzky Universität, Oldenburg, Germany .
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Saadatmand, M.
    Swedish Institute of Computer Science, Sweden .
    Simulation-Based Safety Testing Brake-by-Wire2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017, 2017, p. 61-64Conference paper (Refereed)
    Abstract [en]

    Mechanical systems in cars are replaced by electronic equivalents. To be authorized for the road, validation that the replacements are at least as good as the old systems is required. For electronic braking systems (brake-by-wire), this goodness translates to safety in terms of maintaining timing constraints. Yet, in the future, the safety of braking maneuvres will depend, not only, on electronic brakes, but also on cooperative driving maneuvres orchestrated among many cars. Connecting both brake-by-wire on the microscopic level with cooperative braking on the macroscopic level allows for determining safety on a broader scale, as both systems feed from the same resource: Time. This paper discusses work-in-progress, introducing and combining two threads: electronic brakes and cooperative braking. Discussing safety on two levels simultaneously motivates connecting a Simulink model of an electronic brake-by-wire system with the traffic simulator SUMO for conducting the required combined validation. How safe is a car in relation to a given maximal braking distance? What is the optimal distribution of reaction time between electronic brakes and cooperative braking? The validation focuses on non-functional safety limited by temporal constraints (translated to braking distance). It can be exploited in an early validation approach to help reduce costs of more expensive real world experimentation. It can also determine the boundaries at which sufficient safety can be guaranteed. 

  • 45.
    Müllner, Nils
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Towards a Coherent Terminology and Taxonomy for Evaluating Safety via Testing2018In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security Companion, QRS-C 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 617-622Conference paper (Refereed)
    Abstract [en]

    One common challenge when partners from industry and academia collaborate to determine safety properties of systems, is that terms have different meanings in different domains. This paper provides a collection of relevant terms and explains it in the context of (but not limited to) testing for non-functional properties. The selected terms are connected in a taxonomy, painting the bigger picture, providing relevant references, and connecting the selected terms.

  • 46.
    Nidhra, Srinivas
    et al.
    Blekinge Institute of Technology.
    Yanamadala, Muralidhar
    Blekinge Institute of Technology.
    Afzal, Wasif
    Bahria University, Islamabad, Pakistan.
    Torkar, Richard
    Chalmers University of Technology and the University of Gothenburg.
    Knowledge transfer challenges and mitigation strategies in global software development — A systematic literature review and industrial validation2013In: International Journal of Information Management, ISSN 0268-4012, E-ISSN 1873-4707, Vol. 33, no 2, p. 333-355Article in journal (Refereed)
    Abstract [en]

    Context: In this article we considered knowledge transfer (KT) in global software development (GSD) from two perspectives, state-of-the-art and state-of-the-practice, in order to identify what are the challenges that hamper the success of KT in global software teams, as well as to find out what are the mitigation strategies that can be used to overcome such challenges. Objectives: The overall aim of this work is to provide a body of knowledge for enabling successful KT in GSD settings. This is achieved by an in-depth understanding of KT challenges and mitigation strategies, both from the perspective of literature and industry. It also identifies the similarities and differences in challenges and strategies gathered from literature studies and industrial experts. Methods: In order to fulfill the aim of the research, we collected data through a systematic literature review (SLR) and conducted interviews with industrial experts. Through the SLR we found 35 primary studies relevant to our objectives. We also conducted eight interviews of experienced industrial professionals from eight different multinational companies world-wide. For analyzing the data we used grounded theory and cross-case analysis. Results: In total, 60 different challenges and 79 unique mitigation strategies are identified from both SLR and interview results. The challenges and mitigation strategies are grouped into three core categories of personnel, project and technology factors, thus giving rise to a conceptualization called as 2PT factors. There are greater numbers of challenges and mitigation strategies in the project and personnel factors, highlighting the complex interplay of project-related and human-intensive issues in GSD projects, while the technology factor plays the role as facilitator in transferring knowledge. The study also maps the mitigation strategies to challenges, which can guide practitioners in their selection of strategies to use for overcoming KT challenges in GSD. Conclusions: We conclude that effective management of project and personnel factors, facilitated by technological factors, are crucial for a successful transfer of knowledge in GSD projects. Thus in future, the researchers and practitioners need to focus on the 2PT factors for ensuring effective KT in GSD settings.

  • 47.
    Stanic, Biljana
    et al.
    Mälardalen University.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. IS (Embedded Systems).
    Process Metrics are not Bad Predictors of Fault Proneness2017In: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017, 2017, p. 493-499, article id 8004363Conference paper (Refereed)
    Abstract [en]

    The correct prediction of faulty modules or classes has a number of advantages such as improving the quality of software and assigning capable development resources to fix such faults. There have been different kinds of fault/defect prediction models proposed in literature, but a great majority of them makes use of static code metrics as independent variables for making predictions. Recently, process metrics have gained a considerable attention as alternative metrics to use for making trust-worthy predictions. The objective of this paper is to investigate different combinations of static code and process metrics for evaluating fault prediction performance. We have used publicly available data sets, along with a frequently used classifier, Naive Bayes, to run our experiments. We have, both statistically and visually, analyzed our experimental results. The statistical analysis showed evidence against any significant difference in fault prediction performances for a variety of different combinations of metrics. This reinforced earlier research results that process metrics are as good as predictors of fault proneness as static code metrics. Furthermore, the visual inspection of box plots revealed that the best set of metrics for fault prediction is a mix of both static code and process metrics. We also presented evidence in support of some process metrics being more discriminating than others and thus making them as good predictors to use.

  • 48.
    Strandberg, Per
    et al.
    Westermo Research and Development, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated System Level Regression Test Prioritization in a Nutshell2017In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, no 4, p. 30-37, article id 7974685Article in journal (Refereed)
    Abstract [en]

    Westermo Research and Development has developed SuiteBuilder, an automated tool to determine an effective ordering of regression test cases. The ordering is based on factors such as fault detection success, the interval since the last execution, and code modifications. SuiteBuilder has enabled Westermo to overcome numerous regression-testing problems, including lack of time to run a complete regression suite, failure to detect bugs in a timely manner, and repeatedly omitted tests. In the tool's first two years of use, reordered test suites finished in the available time, most fault-detecting test cases were located in the first third of suites, no important test case was omitted, and the necessity for manual work on the suites decreased greatly. 

  • 49.
    Strandberg, Per Erik
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Westermo R & D AB, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Decision Making and Visualizations Based on Test Results2018In: Empirical Software Engineering and Measurement, 12th International Symposium on ESEM18, 2018Conference paper (Refereed)
    Abstract [en]

    Background: Testing is one of the main methods for quality assurance in the development of embedded software, as well as in software engineering in general. Consequently, test results (and how they are reported and visualized) may substantially influence business decisions in software-intensive organizations. Aims: This case study examines the role of test results from automated nightly software testing and the visualizations for decision making they enable at an embedded systems company in Sweden. In particular, we want to identify the use of the visualizations for supporting decisions from three aspects: in daily work, at feature branch merge, and at release time. Method: We conducted an embedded case study with multiple units of analysis by conducting interviews, questionnaires, using archival data and participant observations. Results: Several visualizations and reports built on top of the test results database are utilized in supporting daily work, merging a feature branch to the master and at release time. Some important visualizations are: lists of failing test cases, easy access to log files, and heatmap trend plots. The industrial practitioners perceived the visualizations and reporting as valuable, however they also mentioned several areas of improvement such as better ways of visualizing test coverage in a functional area as well as better navigation between different views. Conclusions: We conclude that visualizations of test results are a vital decision making tool for a variety of roles and tasks in embedded software development, however the visualizations need to be continuously improved to keep their value for its stakeholders.

  • 50.
    Strandberg, Per Erik
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Westermo Reserarch and Development AB.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Feldt, Robert
    Information Flow in Software Testing: An Interview Study with Embedded Software Engineering PractitionersManuscript (preprint) (Other academic)
    Abstract [en]

    Activities in software testing is a challenge for companies that develop embedded systems where multiple functional teams and technologically difficult tasks are common. This study aims at exploring the information flow in software testing, the perceived challenges and good approaches, for a more effective information flow. We conducted semi-structured interviews with twelve software practitioners working at five organizations in the embedded software industry in Sweden. The interviews were analyzed by means of thematic analysis. The data was classified into six themes that affect the information flow in software testing: testing and troubleshooting, communication, processes, technology, artifacts and organization. We further identified a number of challenges such as poor feedback and understanding exactly what has been tested; and approaches such as fast feedback as well as custom automated test reporting; to achieve an improved information flow. Our results indicate that there are many opportunities to improve this information flow: a first mitigation step is to better understand the challenges and approaches. Future work is needed to realize this in practice, for example to shorten feedback cycles between roles, as well as enhance exploration and visualization of test results

12 1 - 50 of 62
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf