https://www.mdu.se/

mdu.sePublikationer
Ändra sökning
Länk till posten
Permanent länk

Direktlänk
Alternativa namn
Publikationer (10 of 33) Visa alla publikationer
Strandberg, P. E., Ostrand, T. J., Weyuker, E., Afzal, W. & Sundmark, D. (2020). Intermittently Failing Tests in the Embedded Systems Domain. In: International Symposium on Software Testing and Analysis ISSTA'20: . Paper presented at International Symposium on Software Testing and Analysis ISSTA'20, 18 Jul 2020, Los Angeles, United States (pp. 337-348).
Öppna denna publikation i ny flik eller fönster >>Intermittently Failing Tests in the Embedded Systems Domain
Visa övriga...
2020 (Engelska)Ingår i: International Symposium on Software Testing and Analysis ISSTA'20, 2020, s. 337-348Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Software testing is sometimes plagued with intermittently failing tests and finding the root causes of such failing tests is often difficult. This problem has been widely studied at the unit testing level for open source software, but there has been far less investigation at the system test level, particularly the testing of industrial embedded systems. This paper describes our investigation of the root causes of intermittently failing tests in the embedded systems domain, with the goal of better understanding, explaining and categorizing the underlying faults. The subject of our investigation is a currently-running industrial embedded system, along with the system level testing that was performed. We devised and used a novel metric for classifying test cases as intermittent. From more than a half million test verdicts, we identified intermittently and consistently failing tests, and identified their root causes using multiple sources. We found that about 1-3% of all test cases were intermittently failing. From analysis of the case study results and related work, we identified nine factors associated with test case intermittence. We found that a fix for a consistently failing test typically removed a larger number of failures detected by other tests than a fix for an intermittent test. We also found that more effort was usually needed to identify fixes for intermittent tests than for consistent tests. An overlap between root causes leading to intermittent and consistent tests was identified. Many root causes of intermittence are the same in industrial embedded systems and open source software. However, when comparing unit testing to system level testing, especially for embedded systems, we observed that the test environment itself is often the cause of intermittence.

Nyckelord
system level test automation, embedded systems, flaky tests, intermittently failing tests, non-deterministic tests
Nationell ämneskategori
Teknik och teknologier Datorsystem
Identifikatorer
urn:nbn:se:mdh:diva-49332 (URN)10.1145/3395363.3397359 (DOI)2-s2.0-85088915813 (Scopus ID)
Konferens
International Symposium on Software Testing and Analysis ISSTA'20, 18 Jul 2020, Los Angeles, United States
Projekt
EXACT - Experimental Analysis of the Coupling Effect Hypothesis in Software TestingThe Volvo chair of vehicular electronics and software architectureITS ESS-H Industrial Graduate School in Reliable Embedded Sensor SystemsMegaMaRt2 - Megamodelling at Runtime (ECSEL/Vinnova)TESTMINE - Mining Test Evolution for Improved Software Regression Test Selection (KKS)ADEPTNESS - Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems
Tillgänglig från: 2020-07-09 Skapad: 2020-07-09 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Strandberg, P. E., Ostrand, T. J., Weyuker, E., Sundmark, D. & Afzal, W. (2018). Automated test mapping and coverage for network topologies. In: ISSTA 2018 - Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis: . Paper presented at 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2018, 16 July 2018 through 21 July 2018 (pp. 73-83). Association for Computing Machinery, Inc
Öppna denna publikation i ny flik eller fönster >>Automated test mapping and coverage for network topologies
Visa övriga...
2018 (Engelska)Ingår i: ISSTA 2018 - Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery, Inc , 2018, s. 73-83Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Communication devices such as routers and switches play a critical role in the reliable functioning of embedded system networks. Dozens of such devices may be part of an embedded system network, and they need to be tested in conjunction with various computational elements on actual hardware, in many different configurations that are representative of actual operating networks. An individual physical network topology can be used as the basis for a test system that can execute many test cases, by identifying the part of the physical network topology that corresponds to the configuration required by each individual test case. Given a set of available test systems and a large number of test cases, the problem is to determine for each test case, which of the test systems are suitable for executing the test case, and to provide the mapping that associates the test case elements (the logical network topology) with the appropriate elements of the test system (the physical network topology). We studied a real industrial environment where this problem was originally handled by a simple software procedure that was very slow in many cases, and also failed to provide thorough coverage of each network's elements. In this paper, we represent both the test systems and the test cases as graphs, and develop a new prototype algorithm that a) determines whether or not a test case can be mapped to a subgraph of the test system, b) rapidly finds mappings that do exist, and c) exercises diverse sets of network nodes when multiple mappings exist for the test case. The prototype has been implemented and applied to over 10,000 combinations of test cases and test systems, and reduced the computation time by a factor of more than 80 from the original procedure. In addition, relative to a meaningful measure of network topology coverage, the mappings achieved an increased level of thoroughness in exercising the elements of each test system.

Ort, förlag, år, upplaga, sidor
Association for Computing Machinery, Inc, 2018
Nyckelord
Network topology, Subgraph isomorphism, Test coverage, Testing, Embedded systems, Mapping, Test facilities, Topology, Communication device, Computational elements, Industrial environments, Physical network topologies, Prototype algorithms, Software testing
Nationell ämneskategori
Elektroteknik och elektronik
Identifikatorer
urn:nbn:se:mdh:diva-40528 (URN)10.1145/3213846.3213859 (DOI)000612724200008 ()2-s2.0-85051515196 (Scopus ID)9781450356992 (ISBN)
Konferens
27th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2018, 16 July 2018 through 21 July 2018
Tillgänglig från: 2018-08-23 Skapad: 2018-08-23 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Weyuker, E. (2018). Individual working memory capacity traced from multivariate pattern classification of EEG spectral power. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS: . Paper presented at 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018; Hawaii Convention Center, Honolulu; United States; 18 July 2018 through 21 July 2018 (pp. 4812-4815). Institute of Electrical and Electronics Engineers Inc.
Öppna denna publikation i ny flik eller fönster >>Individual working memory capacity traced from multivariate pattern classification of EEG spectral power
2018 (Engelska)Ingår i: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Institute of Electrical and Electronics Engineers Inc. , 2018, s. 4812-4815Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Working Memory (WM) processing is central for human cognitive behavior. Using neurofeedback training to enhance the individual WM capacity is a promising technique but requires careful consideration when choosing the feedback signal. Feedback in terms of univariate spectral power (specifically theta and alpha power) has yielded questionable behavioral effects. However, a promising new direction for WM neurofeedback training is by using a measure of WM that is extracted by multivariate pattern classification. This study recorded EEG oscillatory activity from 15 healthy participants while they were engaged in the n-back task, n[1,2]. Univariate measures of the theta, alpha, and theta-over-alpha power ratio and a measure of WM extracted from multivariate pattern classification (of n-back task load conditions) was compared in relation to individual n-back task performance. Results show that classification performance is positively correlated to individual 2-back task performance while theta, alpha and thetaover-alpha power ratio is not. These results suggest that the discriminability of multivariate EEG oscillatory patterns between two WM load conditions reflects individual WM capacity. 

Ort, förlag, år, upplaga, sidor
Institute of Electrical and Electronics Engineers Inc., 2018
Nationell ämneskategori
Medicinteknik
Identifikatorer
urn:nbn:se:mdh:diva-41851 (URN)10.1109/EMBC.2018.8513130 (DOI)2-s2.0-85056645033 (Scopus ID)9781538636466 (ISBN)
Konferens
40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018; Hawaii Convention Center, Honolulu; United States; 18 July 2018 through 21 July 2018
Tillgänglig från: 2018-12-28 Skapad: 2018-12-28 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Strandberg, P., Afzal, W., Ostrand, T. J., Weyuker, E. & Sundmark, D. (2017). Automated System Level Regression Test Prioritization in a Nutshell. IEEE Software, 34(4), 30-37, Article ID 7974685.
Öppna denna publikation i ny flik eller fönster >>Automated System Level Regression Test Prioritization in a Nutshell
Visa övriga...
2017 (Engelska)Ingår i: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, nr 4, s. 30-37, artikel-id 7974685Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Westermo Research and Development has developed SuiteBuilder, an automated tool to determine an effective ordering of regression test cases. The ordering is based on factors such as fault detection success, the interval since the last execution, and code modifications. SuiteBuilder has enabled Westermo to overcome numerous regression-testing problems, including lack of time to run a complete regression suite, failure to detect bugs in a timely manner, and repeatedly omitted tests. In the tool's first two years of use, reordered test suites finished in the available time, most fault-detecting test cases were located in the first third of suites, no important test case was omitted, and the necessity for manual work on the suites decreased greatly. 

Nationell ämneskategori
Datorsystem
Identifikatorer
urn:nbn:se:mdh:diva-35494 (URN)10.1109/MS.2017.92 (DOI)000405677800008 ()2-s2.0-85024494415 (Scopus ID)
Projekt
ITS ESS-H Industrial Graduate School in Reliable Embedded Sensor SystemsTESTMINE - Mining Test Evolution for Improved Software Regression Test Selection (KKS)
Tillgänglig från: 2017-05-31 Skapad: 2017-05-31 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Weyuker, E. & Ostrand, T. J. (2017). Experiences with academic-industrial collaboration on empirical studies of software systems. In: 2017 IEEE 28TH INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING WORKSHOPS (ISSREW 2017): . Paper presented at 28th IEEE International Symposium on Software Reliability Engineering (pp. 164-168). Institute of Electrical and Electronics Engineers Inc.
Öppna denna publikation i ny flik eller fönster >>Experiences with academic-industrial collaboration on empirical studies of software systems
2017 (Engelska)Ingår i: 2017 IEEE 28TH INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING WORKSHOPS (ISSREW 2017), Institute of Electrical and Electronics Engineers Inc. , 2017, s. 164-168Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The authors have held both academic and industrial research positions, and have designed and carried out many empirical studies of large software systems that were built and maintained in industrial environments. Their experiences show that the most crucial component of a successful study is the participation of at least one industrial collaborator who is committed to the study’s goals and is able to provide advice and assistance throughout the course of the study. This paper describes studies carried out in three different industrial environments, discusses obstacles that arise, and how the authors have been able to overcome some of those obstacles. 

Ort, förlag, år, upplaga, sidor
Institute of Electrical and Electronics Engineers Inc., 2017
Nyckelord
Computer software; Industrial research; Technical presentations, Collaboration; Data deficiency; Data quality; Empirical studies; Proprietary data, Software reliability
Nationell ämneskategori
Datorsystem
Identifikatorer
urn:nbn:se:mdh:diva-38580 (URN)10.1109/ISSREW.2017.12 (DOI)000418465000045 ()2-s2.0-85040607239 (Scopus ID)
Konferens
28th IEEE International Symposium on Software Reliability Engineering
Tillgänglig från: 2018-02-08 Skapad: 2018-02-08 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Landwehr, C., Ludewig, J., Meersman, R., Parnas, D. L., Shoval, P., Wand, Y., . . . Weyuker, E. (2017). Software Systems Engineering programmes a capability approach. Journal of Systems and Software, 125, 354-364
Öppna denna publikation i ny flik eller fönster >>Software Systems Engineering programmes a capability approach
Visa övriga...
2017 (Engelska)Ingår i: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 125, s. 354-364Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper discusses third-level educational programmes that are intended to prepare their graduates for a career building systems in which software plays a major role. Such programmes are modelled on traditional Engineering programmes but have been tailored to applications that depend heavily on software. Rather than describe knowledge that should be taught, we describe capabilities that students should acquire in these programmes. The paper begins with some historical observations about the software development field. 

Nationell ämneskategori
Datorsystem
Identifikatorer
urn:nbn:se:mdh:diva-35175 (URN)10.1016/j.jss.2016.12.016 (DOI)000395359500022 ()2-s2.0-85008352097 (Scopus ID)
Tillgänglig från: 2017-04-13 Skapad: 2017-04-13 Senast uppdaterad: 2018-03-05Bibliografiskt granskad
Enoiu, E. P., Causevic, A., Ostrand, T., Weyuker, E., Sundmark, D. & Pettersson, P. (2016). Automated Test Generation using Model-Checking: An Industrial Evaluation. International Journal on Software Tools for Technology Transfer, 18(3), 335-353
Öppna denna publikation i ny flik eller fönster >>Automated Test Generation using Model-Checking: An Industrial Evaluation
Visa övriga...
2016 (Engelska)Ingår i: International Journal on Software Tools for Technology Transfer, ISSN 1433-2779, E-ISSN 1433-2787, Vol. 18, nr 3, s. 335-353Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model-checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state-space explosion can, however, impact model-checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model-checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox COMPLETETEST can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.

Ort, förlag, år, upplaga, sidor
Germany: Springer, 2016
Nyckelord
test generation, model checking, automated software testing, function block diagram, safety-critical software, industrial study
Nationell ämneskategori
Elektroteknik och elektronik
Identifikatorer
urn:nbn:se:mdh:diva-28100 (URN)10.1007/s10009-014-0355-9 (DOI)000374974200007 ()2-s2.0-84911400022 (Scopus ID)1433-2779 (ISBN)
Projekt
ATAC - Advanced Test Automation for Complex Software-Intensive System (ITEA2/Vinnova)ITS-EASY Post Graduate School for Embedded Software and Systems
Tillgänglig från: 2015-06-08 Skapad: 2015-06-08 Senast uppdaterad: 2024-01-17Bibliografiskt granskad
Derehag, J., Weyuker, E., Ostrand, T. & Sundmark, D. (2016). Transitioning Fault Prediction Models to a New Environment. In: Proceedings - 2016 12th European Dependable Computing Conference, EDCC 2016: . Paper presented at 12th European Dependable Computing Conference (EDCC), SEP 05-09, 2016, Gothenburg, SWEDEN (pp. 241-248). , Article ID 7780365.
Öppna denna publikation i ny flik eller fönster >>Transitioning Fault Prediction Models to a New Environment
2016 (Engelska)Ingår i: Proceedings - 2016 12th European Dependable Computing Conference, EDCC 2016, 2016, s. 241-248, artikel-id 7780365Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We describe the application and evaluation of fault prediction algorithms to a project developed by a Swedish company that transitioned from waterfall to agile development methods. The project used two different version control systems and a separate bug tracking system during its lifetime. The algorithms were originally designed for use on systems implemented with a traditional waterfall process at American companies that maintained their project records in an integrated database system that combined bug recording and version control. We compare the performance of the original prediction model on the American systems to the results obtained in the Swedish environment in both its pre-agile and agile stages. We also consider the impact of additional variables in the model.

Nyckelord
faults, defects, fault prediction, agile, case study
Nationell ämneskategori
Datorsystem
Identifikatorer
urn:nbn:se:mdh:diva-34665 (URN)10.1109/EDCC.2016.21 (DOI)000390696300026 ()2-s2.0-85013845472 (Scopus ID)978-1-5090-1582-5 (ISBN)
Konferens
12th European Dependable Computing Conference (EDCC), SEP 05-09, 2016, Gothenburg, SWEDEN
Tillgänglig från: 2017-01-19 Skapad: 2017-01-19 Senast uppdaterad: 2022-11-09Bibliografiskt granskad
Ostrand, T. & Weyuker, E. (2013). Predicting Bugs in Large Industrial Software Systems. In: Software Engineering: International Summer Schools, ISSSE 2009-2011, Salerno, Italy. Revised Tutorial Lectures (pp. 71-93). Germany: Springer Berlin/Heidelberg
Öppna denna publikation i ny flik eller fönster >>Predicting Bugs in Large Industrial Software Systems
2013 (Engelska)Ingår i: Software Engineering: International Summer Schools, ISSSE 2009-2011, Salerno, Italy. Revised Tutorial Lectures, Germany: Springer Berlin/Heidelberg, 2013, s. 71-93Kapitel i bok, del av antologi (Refereegranskat)
Abstract [en]

This chapter is a survey of close to ten years of software fault prediction research performed by our group. We describe our initial motivation, the variables used to make predictions, provide a description of our standard model based on Negative Binomial Regression, and summarize the results of using this model to make predictions for nine large industrial software systems. The systems range in size from hundreds of thousands to millions of lines of code. All have been in the field for multiple years and many releases, and continue to be maintained and enhanced, usually at 3 month intervals. Effectiveness of the fault predictions is assessed using two different metrics. We compare the effectiveness of the standard model to augmented models that include variables related to developer counts, to inter-file calling structure, and to information about specific developers who modified the code. We also evaluate alternate prediction models based on different training algorithms, including Recursive Partitioning, Bayesian Additive Regression Trees, and Random Forests.

Ort, förlag, år, upplaga, sidor
Germany: Springer Berlin/Heidelberg, 2013
Serie
Lecture Notes in Computer Science, ISSN 0302-9743 ; 7171
Nationell ämneskategori
Teknik och teknologier
Identifikatorer
urn:nbn:se:mdh:diva-23705 (URN)10.1007/978-3-642-36054-1_3 (DOI)2-s2.0-84893794271 (Scopus ID)978-3-642-36053-4 (ISBN)
Tillgänglig från: 2013-12-18 Skapad: 2013-12-18 Senast uppdaterad: 2014-04-24Bibliografiskt granskad
Bell, R., Ostrand, T. J. & WEYUKER, E. (2013). The limited impact of individual developer data on software defect prediction. Empirical Software Engineering, 18(3), 478-505
Öppna denna publikation i ny flik eller fönster >>The limited impact of individual developer data on software defect prediction
2013 (Engelska)Ingår i: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 18, nr 3, s. 478-505Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Previous research has provided evidence that a combination of static code metrics and software history metrics can be used to predict with surprising success which files in the next release of a large system will havethe largest numbers of defects. In contrast, very little research exists to indicate whether information about individual developers can profitably be used to improve predictions. We investigate whether files in a large system that are modified by an individual developer consistently contain either more or fewer faults than the average of all files in the system. The goal of the investigation is to determine whether information about which particular developer modified a file is able to improve defect predictions. We also extend earlier research evaluating use of counts of the number of developers who modified a file as predictors of the file's future faultiness. We analyze change reports filed for three large systems, each containing 18 releases, with a combined total of nearly 4 million LOC and over 11,000 files. A buggy file ratio is defined for programmers, measuring the proportion of faulty files in Release R out of all files modified by the programmer in Release R-1. We assess the consistency of the buggy file ratio across releases for individual programmers both visually and within the context of a fault prediction model. Buggy file ratios for individual programmers often varied widely across all the releases that they participated in. A prediction model that takes account of the history of faulty files that were changed by individual developers shows improvement over the standard negative binomial model of less than 0.13% according to one measure, and no improvement at all according to another measure. In contrast, augmenting a standard model with counts of cumulative developers changing files in prior releases produced up to a 2% improvement in the percentage of faults detected in the top 20% of predicted faulty files. The cumulative number of developers interacting with a file can be a useful variable for defect prediction. However, the study indicates that adding information to a model about which particular developermodified a file is not likely to improve defect predictions.

Nationell ämneskategori
Teknik och teknologier
Identifikatorer
urn:nbn:se:mdh:diva-23675 (URN)10.1007/s10664-011-9178-4 (DOI)000319774300003 ()2-s2.0-84878737201 (Scopus ID)
Tillgänglig från: 2013-12-17 Skapad: 2013-12-17 Senast uppdaterad: 2022-09-15Bibliografiskt granskad
Organisationer
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0002-1660-199X

Sök vidare i DiVA

Visa alla publikationer