mdh.sePublications
Change search
Refine search result
1 - 32 of 32
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Avritzer, A
    et al.
    Siemens Corporate Research, USA.
    Tanikella, R
    Siemens Corporate Research, USA.
    James, K
    Siemens Corporate Research, USA.
    Cole, R
    JHU, Applied Physics Laboratory, USA.
    weyuker, elaine
    AT and T Labs, USA.
    Monitoring for Security Intrusion using Performance Signatures2010In: WOSP/SIPEW'10 - Proceedings of the 1st Joint WOSP/SIPEW International Conference on Performance Engineering, 2010, p. 93-103Conference paper (Refereed)
    Abstract [en]

    A new approach for detecting security attacks on software systems by monitoring the software system performance signatures is introduced. We present a proposed architecture for security intrusion detection using off-the-shelf security monitoring tools and performance signatures. Our approach relies on the assumption that the performance signature of the well-behaved system can be measured and that the performancesignature of several types of attacks can be identified. This assumption has been validated for operations support systems that are used to monitor large infrastructures and receive aggregated traffic that is periodic in nature. Examples of such infrastructures include telecommunications systems, transportation systems and power generation systems. In addition, significant deviation from well-behaved system performance signatures can be used to trigger alerts about new types of security attacks. We used a custom performance benchmark and five types of security attacks to deriveperformance signatures for the normal mode of operation and the security attack mode of operation. We observed that one of the types of thesecurity attacks went undetected by the off-the-shelf security monitoring tools but was detected by our approach of monitoring performance signatures. We conclude that an architecture for security intrusion detection can be effectively complemented by monitoring of performance signatures.

  • 2.
    Avritzer, A
    et al.
    Siemens Corporate Research, USA.
    Weyuker, Elaine
    AT and T Labs - Research, USA.
    The Automated Generation of Test Cases using an Extended Domain Based Reliability Model2009In: Proceedings of the 2009 ICSE Workshop on Automation of Software Test, AST 2009, 2009, p. 44-52, article id 5069040Conference paper (Refereed)
  • 3.
    Avritzer, Alberto
    et al.
    Siemens Corporate Research, United States .
    Cole, R
    JHU/Applied Physics Laboratory, United States .
    Weyuker, Elaine
    AT and T Labs - Research, United States.
    Methods and Opportunities for Rejuvenation in Aging Distributed Software2010In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 83, no 9, p. 1568-1578Article in journal (Refereed)
    Abstract [en]

    In this paper we describe several methods for detecting the need for software rejuvenation in mission critical systems that are subjected to worm infection, and introduce new software rejuvenation algorithms. We evaluate these algorithms' effectiveness using both simulation studies and analytic modeling, by assessing the probability of mission success. The system under study emulates a Mobile Ad-Hoc Network (MANET) of processing nodes. Our analysis determined that some of our rejuvenation algorithms are quite effective in maintaining a high probability of mission success while the system is under explicit attack by a worm infection.

  • 4.
    Avritzer, Alberto
    et al.
    Siemens Corporate Research, United States .
    Rosa Maria, Meri Ledmundo de Souza e Silvaa
    Federal University of Rio de Janeiro, Brazil .
    Leao, RMM
    Federal University of Rio de Janeiro, Brazil .
    weyuker, elaine
    Mälardalen University, School of Innovation, Design and Engineering.
    Generating Test Cases Using using a Performability Model2011In: IET Software, ISSN 1751-8806, E-ISSN 1751-8814, Vol. 5, no 2, p. 113-119Article in journal (Refereed)
    Abstract [en]

    The authors present a new approach for the automated generation of test cases to be used for demonstrating the reliability of large industrial mission-critical systems. In this study they extend earlier work by using a performability model to track resource usage and resource failures. Results from the transient Markov chain analysis are used to estimate the software reliability at a given system execution time.

  • 5. Bell, R
    et al.
    Ostrand, T
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering.
    Does Measuring Code Change Improve Fault Prediction?2011In: ACM International Conference Proceeding Series, 2011Conference paper (Refereed)
  • 6.
    Bell, R
    et al.
    AT and T Labs Research, United States .
    Ostrand, Thomas J.
    AT and T Labs Research, United States .
    WEYUKER, ELAINE
    AT and T Labs Research, United States .
    The limited impact of individual developer data on software defect prediction2013In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 18, no 3, p. 478-505Article in journal (Refereed)
    Abstract [en]

    Previous research has provided evidence that a combination of static code metrics and software history metrics can be used to predict with surprising success which files in the next release of a large system will havethe largest numbers of defects. In contrast, very little research exists to indicate whether information about individual developers can profitably be used to improve predictions. We investigate whether files in a large system that are modified by an individual developer consistently contain either more or fewer faults than the average of all files in the system. The goal of the investigation is to determine whether information about which particular developer modified a file is able to improve defect predictions. We also extend earlier research evaluating use of counts of the number of developers who modified a file as predictors of the file's future faultiness. We analyze change reports filed for three large systems, each containing 18 releases, with a combined total of nearly 4 million LOC and over 11,000 files. A buggy file ratio is defined for programmers, measuring the proportion of faulty files in Release R out of all files modified by the programmer in Release R-1. We assess the consistency of the buggy file ratio across releases for individual programmers both visually and within the context of a fault prediction model. Buggy file ratios for individual programmers often varied widely across all the releases that they participated in. A prediction model that takes account of the history of faulty files that were changed by individual developers shows improvement over the standard negative binomial model of less than 0.13% according to one measure, and no improvement at all according to another measure. In contrast, augmenting a standard model with counts of cumulative developers changing files in prior releases produced up to a 2% improvement in the percentage of faults detected in the top 20% of predicted faulty files. The cumulative number of developers interacting with a file can be a useful variable for defect prediction. However, the study indicates that adding information to a model about which particular developermodified a file is not likely to improve defect predictions.

  • 7.
    Bell, R
    et al.
    AT and T Labs, USA.
    weyuker, elaine
    AT and T Labs, USA.
    Ostrand, T
    AT and T Labs, USA.
    Assessing the Impact of Using Fault-Prediction in Industry2011In: Proceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011, 2011, p. 561-565Conference paper (Refereed)
    Abstract [en]

    Do fault prediction models that guide testing and other efforts to improve software reliability lead to finding different or additional faults in the next release, to an improved process for finding the same faults that would occur were the models not used, or do they have no impact at all? In this challenge paper, we describe the difficulties involved in estimating effects of this sort of intervention and discuss ways to empirically answer that question and ways of assessing any changes, if present. We present several experimental design options and discuss the pros and cons of each.

  • 8.
    Derehag, Jesper
    et al.
    Ericsson AB, Gothenburg, Sweden..
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Transitioning Fault Prediction Models to a New Environment2016In: Proceedings - 2016 12th European Dependable Computing Conference, EDCC 2016, 2016, p. 241-248, article id 7780365Conference paper (Refereed)
    Abstract [en]

    We describe the application and evaluation of fault prediction algorithms to a project developed by a Swedish company that transitioned from waterfall to agile development methods. The project used two different version control systems and a separate bug tracking system during its lifetime. The algorithms were originally designed for use on systems implemented with a traditional waterfall process at American companies that maintained their project records in an integrated database system that combined bug recording and version control. We compare the performance of the original prediction model on the American systems to the results obtained in the Swedish environment in both its pre-agile and agile stages. We also consider the impact of additional variables in the model.

  • 9.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Software Engineering Research Consultant, Västerås, Sweden.
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Swedish Institute of Computer Science, Stockholm, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated Test Generation using Model-Checking: An Industrial Evaluation2016In: International Journal on Software Tools for Technology Transfer (STTT), ISSN 1433-2779, E-ISSN 1433-2787, Vol. 18, no 3, p. 335-353Article in journal (Refereed)
    Abstract [en]

    In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model-checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state-space explosion can, however, impact model-checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model-checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox COMPLETETEST can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.

  • 10.
    Landwehr, Carl
    et al.
    George Washington Univ, Washington DC, USA.
    Ludewig, Jochen
    Univ Stuttgart, Stuttgart, Germany.
    Meersman, Robert
    Graz Univ Technol, Austria.
    Parnas, David Lorge
    Middle Rd Software, Ottawa, ON, Canada.
    Shoval, Peretz
    Ben Gurion Univ Negev, Beer Sheva, Israel.
    Wand, Yair
    Univ British Columbia, Vancouver, BC, Canada.
    Weiss, David
    Iowa State Univ, Ames, IA USA.
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Univ Cent Florida, Orlando, USA.
    Software Systems Engineering programmes a capability approach2017In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 125, p. 354-364Article in journal (Refereed)
    Abstract [en]

    This paper discusses third-level educational programmes that are intended to prepare their graduates for a career building systems in which software plays a major role. Such programmes are modelled on traditional Engineering programmes but have been tailored to applications that depend heavily on software. Rather than describe knowledge that should be taught, we describe capabilities that students should acquire in these programmes. The paper begins with some historical observations about the software development field. 

  • 11.
    Ostrand, T
    et al.
    AT and T Labs - Research, USA.
    Weyuker, Elaine
    AT and T Labs - Research, USA.
    Can File Level Characteristics Help Identify System Level Fault-Proneness?2011In: Hardware and Software: Verification and Testing: 7th International Haifa Verification Conference, HVC 2011, Haifa, Israel, December 6-8, 2011, Revised Selected Papers, Springer, 2011, p. 176-189Chapter in book (Refereed)
    Abstract [en]

    In earlier studies of multiple-release systems, we observed that the number of changes and the number of faults in a file in the past release, the size of a file, and the maturity of a file are all useful predictors of the file's fault proneness in the next release. In each case the data needed to make predictions have been extracted from a configuration management system which provides integrated change management and version control functionality. In this paper we investigate analogous questions for the system as a whole, rather than looking at its constituent files. Using two large industrial software systems, each with many field releases, we examine a number of questions relating defects to system maturity, how often the system has changed, the size difference of a release from the prior release, and the length of time a release has been under development before the start of system testing. Most of our observations match neither our intuition, nor the relations observed for these two systems when similar questions were asked at the file level.

  • 12.
    Ostrand, T
    et al.
    AT and T Labs, USA.
    Weyuker, Elaine
    AT and T Labs, USA.
    Predicting Bugs in Large Industrial Software Systems2013In: Software Engineering: International Summer Schools, ISSSE 2009-2011, Salerno, Italy. Revised Tutorial Lectures, Germany: Springer Berlin/Heidelberg, 2013, p. 71-93Chapter in book (Refereed)
    Abstract [en]

    This chapter is a survey of close to ten years of software fault prediction research performed by our group. We describe our initial motivation, the variables used to make predictions, provide a description of our standard model based on Negative Binomial Regression, and summarize the results of using this model to make predictions for nine large industrial software systems. The systems range in size from hundreds of thousands to millions of lines of code. All have been in the field for multiple years and many releases, and continue to be maintained and enhanced, usually at 3 month intervals. Effectiveness of the fault predictions is assessed using two different metrics. We compare the effectiveness of the standard model to augmented models that include variables related to developer counts, to inter-file calling structure, and to information about specific developers who modified the code. We also evaluate alternate prediction models based on different training algorithms, including Recursive Partitioning, Bayesian Additive Regression Trees, and Random Forests.

  • 13.
    Ostrand, T
    et al.
    AT and T Labs - Research, USA.
    Weyuker, Elaine
    AT and T Labs - Research, USA.
    Progress in Automated Software Defect Prediction2008In: Lecture Notes in Computer Science, v. 5394, 2008, p. 200-204Conference paper (Refereed)
  • 14.
    Shin, Y
    et al.
    North Carolina State University, United States .
    Bell, R
    AT and T Labs Research, United States .
    Ostrand, T
    AT and T Labs Research, United States .
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering. AT and T Labs Research, United States .
    On the use of calling structure information to improve fault prediction2012In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 17, no 4-5, p. 390-423Article in journal (Refereed)
    Abstract [en]

    Previous studies have shown that software code attributes, such as lines of source code, and history information, such as the number of code changes and the number of faults in prior releases of software, are useful for predicting where faults will occur. In this study of two large industrial software systems, we investigate the effectiveness of adding information about calling structure to fault prediction models. Adding callingstructure information to a model based solely on non-calling structure code attributes modestly improved prediction accuracy. However, the addition of calling structure information to a model that included both history and non-calling structure code attributes produced no improvement.

  • 15.
    Strandberg, Per
    et al.
    Westermo Research and Development, Sweden.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated System Level Regression Test Prioritization in a Nutshell2017In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, no 4, p. 30-37, article id 7974685Article in journal (Refereed)
    Abstract [en]

    Westermo Research and Development has developed SuiteBuilder, an automated tool to determine an effective ordering of regression test cases. The ordering is based on factors such as fault detection success, the interval since the last execution, and code modifications. SuiteBuilder has enabled Westermo to overcome numerous regression-testing problems, including lack of time to run a complete regression suite, failure to detect bugs in a timely manner, and repeatedly omitted tests. In the tool's first two years of use, reordered test suites finished in the available time, most fault-detecting test cases were located in the first third of suites, no important test case was omitted, and the necessity for manual work on the suites decreased greatly. 

  • 16.
    Strandberg, Per Erik
    et al.
    Westermo RandD AB, Sweden.
    Ostrand, Thomas J.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    WEYUKER, ELAINE
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated test mapping and coverage for network topologies2018In: ISSTA 2018 - Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery, Inc , 2018, p. 73-83Conference paper (Refereed)
    Abstract [en]

    Communication devices such as routers and switches play a critical role in the reliable functioning of embedded system networks. Dozens of such devices may be part of an embedded system network, and they need to be tested in conjunction with various computational elements on actual hardware, in many different configurations that are representative of actual operating networks. An individual physical network topology can be used as the basis for a test system that can execute many test cases, by identifying the part of the physical network topology that corresponds to the configuration required by each individual test case. Given a set of available test systems and a large number of test cases, the problem is to determine for each test case, which of the test systems are suitable for executing the test case, and to provide the mapping that associates the test case elements (the logical network topology) with the appropriate elements of the test system (the physical network topology). We studied a real industrial environment where this problem was originally handled by a simple software procedure that was very slow in many cases, and also failed to provide thorough coverage of each network's elements. In this paper, we represent both the test systems and the test cases as graphs, and develop a new prototype algorithm that a) determines whether or not a test case can be mapped to a subgraph of the test system, b) rapidly finds mappings that do exist, and c) exercises diverse sets of network nodes when multiple mappings exist for the test case. The prototype has been implemented and applied to over 10,000 combinations of test cases and test systems, and reduced the computation time by a factor of more than 80 from the original procedure. In addition, relative to a meaningful measure of network topology coverage, the mappings achieved an increased level of thoroughness in exercising the elements of each test system.

  • 17.
    Strandberg, Per Erik
    et al.
    Westermo Research and Development AB, Västerås, Sweden.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Experience Report: Automated System Level Regression Test Prioritization Using Multiple Factors2016In: 27th International Symposium on Software Reliability Engineering ISSRE'16, 2016Conference paper (Refereed)
    Abstract [en]

    We propose a new method of determining an effective ordering of regression test cases, and describe its implementation as an automated tool called SuiteBuilder developed by Westermo Research and Development AB. The tool generates an efficient order to run the cases in an existing test suite by using expected or observed test duration and combining priorities of multiple factors associated with test cases, including previous fault detection success, interval since last executed, and modifications to the code tested. The method and tool were developed to address problems in the traditional process of regression testing, such as lack of time to run a complete regression suite, failure to detect bugs in time, and tests that are repeatedly omitted. The tool has been integrated into the existing nightly test framework for Westermo software that runs on large-scale data communication systems.  In experimental evaluation of the tool, we found significant improvement in regression testing results. The re-ordered test suites finish within the available time, the majority of fault-detecting test cases are located in the first third of the suite, no important test case is omitted, and the necessity for manual work on the suites is greatly reduced.

  • 18.
    Weyuker, Elaine
    AT&T Labs - Research.
    Comparing the Effectiveness of Testing Techniques2008In: Formal Methods and Testing: An Outcome of the FORTEST Network, Revised Selected Papers / [ed] Robert M. Hierons et. al., Germany: Springer, 2008, p. 271-297Chapter in book (Refereed)
    Abstract [en]

    Testing software systems requires practitioners to decide how to select test data. This chapter discusses what it means for one test data selection criterion to be more effective than another. Several proposed comparison relations are discussed, highlighting the strengths and weaknesses of each. Also included is a discussion of how these relations evolved and argue that large scale empirical studies are needed.

  • 19.
    Weyuker, Elaine
    AT and T Labs - Research, USA.
    Empirical Software Engineering Research - The Good, The Bad, The Ugly2011In: International Symposium on Empirical Software Engineering and Measurement, 2012, 2011, p. -Article number 6092548Conference paper (Refereed)
    Abstract [en]

    The Software Engineering Research community has slowly recognized that empirical studies are an important way of validating ideas and increasingly our community has stopped accepting the sufficiency of arguing that a smart person has come up with the idea and therefore it must be good. This has led to a flood of Software Engineering papers that contain at least some form of empirical study. However, not all empirical studies are created equal, and many may not even provide any useful information or value. We survey the gradual shift from essentially no empirical studies, to a small number of ones of questionable value, and look at what we need to do to insure that our empirical studies really contribute to the state of knowledge in the field. Thus we have the good, the bad, and the ugly. What are we as a community doing correctly? What are we doing less well than we should be because we either don't have the necessary artifacts or because the time and resources required to do "the good" is perceived to be too great? And where are we missing the boat entirely in terms of not addressing critical questions and often not even recognizing that these questions are central even if we don't know the answers. We look to see whether we can find some commonality in the projects that have really made the transition from research to widespread practice to see whether we can identify some common themes.

  • 20.
    weyuker, elaine
    et al.
    AT and T Labs - Research, United States.
    Avritzer, A.
    Siemens Corporate Research, United States.
    Cole, R
    JHU/Applied Physics Laboratory, United States.
    Methods and Opportunities for Rejuvenation in Aging Distributed Software Systems2008Conference paper (Refereed)
  • 21.
    weyuker, elaine
    et al.
    AT and T Labs, USA.
    Bell, R
    AT and T Labs, USA.
    Ostrand, T
    AT and T Labs, USA.
    Replicate, Replicate, Replicate2011Conference paper (Refereed)
    Abstract [en]

    Replication is a standard part of scientific experimentation. Unfortunately, in software engineering, replication of experiments is often considered an inferior type of research, or not even research at all. In this paper we describe four different types of replication that we have been performing as part of validating the effectiveness and applicability of our software fault prediction research. We discuss replication over time, replication by using different subject systems, replication by changing the variables in prediction models, and replication by varying the modeling algorithms.

  • 22.
    Weyuker, Elaine
    et al.
    AT and T Labs - Research, USA.
    Bell, R
    AT and T Labs - Research, USA.
    Ostrand, T
    AT and T Labs - Research, USA.
    We’re Finding Most of the Bugs, but What Are We Missing2010In: ICST 2010 - 3rd International Conference on Software Testing, Verification and Validation, 2010, p. 313-322, article id 5477073Conference paper (Refereed)
    Abstract [en]

    We compare two types of model that have been used to predict software fault-proneness in the next release of a software system. Classification models make a binary prediction that a software entity such as a file or module is likely to be either faulty or not faulty in the next release. Ranking models order the entities according to their predicted number of faults. They are generally used to establish a priority for more intensive testing of the entities that occur early in the ranking. We investigate ways of assessing both classification models and ranking models, and the extent to which metrics appropriate for one type of model are also appropriate for the other. Previous work has shown that ranking models are capable of identifying relatively small sets of files that contain 75-95% of the faults detected in the next release of large legacy systems. In our studies of the rankings produced by these models, the faults not contained in the predicted most faultprone files are nearly always distributed across many of the remaining files; i.e., a single file that is in the lower portion of the ranking virtually never contains a large number of faults.

  • 23.
    Weyuker, Elaine
    et al.
    AT and T Labs - Research, USA.
    Ostrand, T
    AT and T Labs - Research, USA.
    Comparing Methods to Identify Defect Reports in a Change Management Database2008In: DEFECTS'08: 2008 International Symposium on Software Testing and Analysis - Proceedings of the 2008 Workshop on Defects in Large Software Systems 2008, DEFECTS'08, 2008, p. 27-31Conference paper (Refereed)
    Abstract [en]

    A key problem when doing automated fault analysis and fault prediction from information in a software change management database is how to determine which change reports represent software faults. In some change management systems, there is no simple way to distinguish fault reports from changes made to add new functionality or perform routine maintenance. This paper describes a comparison of two methods for classifying change reports for a large software system, and concludes that, for that particular system, the stage of development when the report was initialized is a more accurate indicator of its fault status than the presence of certain keywords in the report's natural language description.

  • 24.
    Weyuker, Elaine
    et al.
    AT and T Labs - Research, USA.
    Ostrand, T
    AT and T Labs - Research, USA.
    Comparing Negative Binomial and Recursive Partitioning Models for Fault Prediction2008In: Proceedings - International Conference on Software Engineering, 2008, 2008, p. 3-9Conference paper (Refereed)
  • 25. Weyuker, Elaine
    et al.
    Ostrand, T
    Software Fault Prediction Tool2010Conference paper (Refereed)
  • 26.
    Weyuker, Elaine
    et al.
    AT&T Labs - Research, USA.
    Ostrand, T
    AT&T Labs - Research, USA.
    What Can Fault Prediction Do For YOU?2008In: Tests and Proofs, Springer, 2008, p. 18-29Conference paper (Refereed)
    Abstract [en]

    It would obviously be very valuable to know in advance which files in the next release of a large software system are most likely to contain the largest numbers of faults. This is true whether the goal is to validate the system by testing or formally verifying it, or by using some hybrid approach. To accomplish this, we developed negative binomial regression models and used them to predict the expected number of faults in each file of the next release of a system. The predictions are based on code characteristics and fault and modification history data. This paper discusses what we have learned from applying the model to several large industrial systems, each with multiple years of field exposure. It also discusses our success in making accurate predictions and some of the issues that had to be considered.

  • 27.
    Weyuker, Elaine
    et al.
    AT and T Labs, United States .
    Ostrand, T
    AT and T Labs, United States .
    Bell, R
    AT and T Labs, United States .
    Comparing the Effectiveness of Several Modeling Methods for Fault Prediction2010In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 15, no 3, p. 277-295Article in journal (Refereed)
    Abstract [en]

    We compare the effectiveness of four modeling methods-negative binomial regression, recursive partitioning, random forests and Bayesian additive regression trees-for predicting the files likely to contain the most faults for 28 to 35 releases of three large industrial software systems. Predictor variables included lines of code, file age, faults in the previous release, changes in the previous two releases, and programming language. To compare the effectiveness of the different models, we use two metrics-the percent of faults contained in the top 20% of files identified by the model, and a new, more general metric, the fault-percentile-average. The negative binomial regression and random forests models performed significantly better than recursive partitioning and Bayesian additive regression trees, as assessed by either of the metrics. For each of the three systems, the negative binomial and random forests models identified 20% of the files in each release that contained an average of 76% to 94% of the faults. 

  • 28.
    weyuker, elaine
    et al.
    AT&T Labs Res, Florham Pk, NJ 07932 USA.
    Ostrand, T
    AT&T Labs Res, Florham Pk, NJ 07932 USA.
    Bell, R
    AT&T Labs Res, Florham Pk, NJ 07932 USA.
    Do Too Many Cooks Spoil the Broth? Using the Number of Developers to Enhance Defect Prediction Models2008In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 13, no 5, p. 539-559Article in journal (Refereed)
    Abstract [en]

    Fault prediction by negative binomial regression models is shown to be effective for four large production software systems from industry. A model developed originally with data from systems with regularly scheduled releases was successfully adapted to a system without releases to identify 20% of that system's files that contained 75% of the faults. A model with a pre-specified set of variables derived from earlier research was applied to three additional systems, and proved capable of identifying averages of 81, 94 and 76% of the faults in those systems. A primary focus of this paper is to investigate the impact on predictive accuracy of using data about the number of developers who access individual code units. For each system, including the cumulative number of developers who had previously modified a file yielded no more than a modest improvement in predictive accuracy. We conclude that while many factors can "spoil the broth" (lead to the release of software with too many defects), the number of developers is not a major influence.

  • 29.
    Weyuker, Elaine
    et al.
    AT and T Labs - Research, USA.
    Ostrand, T.
    AT and T Labs - Research, USA.
    Bell, R. M.
    AT and T Labs - Research, USA.
    Programmer-based Fault Prediction2010In: ACM International Conference Proceeding Series, 2010, article id 19Conference paper (Refereed)
    Abstract [en]

    Background: Previous research has provided evidence that a combination of static code metrics and software history metrics can be used to predict with surprising success which files in the next release of a large system will have the largest numbers of defects. In contrast, very little research exists to indicate whether information about individual developers can profitably be used to improve predictions. Aims: We investigate whether files in a large system that are modified by an individual developer consistently contain either more or fewer faults than the average of all files in the system. The goal of the investigation is to determine whether information about which particular developer modified a file is able to improve defect predictions. We also continue an earlier study to evaluate the use of counts of the number of developers who modified a file as predictors of the file's future faultiness. Method: We analyzed change reports filed by 107 programmers for 16 releases of a system with 1,400,000 LOC and 3100 files. A "bug ratio" was defined for programmers, measuring the proportion of faulty files in release R out of all files modified by the programmer in release R-1. The study compares the bug ratios of individual programmers to the average bug ratio, and also assesses the consistency of the bug ratio across releases for individual programmers. Results: Bug ratios varied widely among all the programmers, as well as for many individual programmers across all the releases that they participated in. We found a statistically significant correlation between the bug ratios for programmers for the first half of changed files versus the ratios for the second half, indicating a measurable degree of persistence in the bug ratio. However, when the computation was repeated with the bug ratio controlled not only by release, but also by file size, the correlation disappeared. In addition to the bug ratios, we confirmed that counts of the cumulative number of different developers changing a file over its lifetime can help to improve predictions, while other developer counts are not helpful.

  • 30.
    WEYUKER, ELAINE
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. University of Central Florida, Orlando, FL, United States.
    Ostrand, Thomas J.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Experiences with academic-industrial collaboration on empirical studies of software systems2017In: Proceedings - 2017 IEEE 28th International Symposium on Software Reliability Engineering Workshops, ISSREW 2017, p. 164-168Article in journal (Refereed)
    Abstract [en]

    The authors have held both academic and industrial research positions, and have designed and carried out many empirical studies of large software systems that were built and maintained in industrial environments. Their experiences show that the most crucial component of a successful study is the participation of at least one industrial collaborator who is committed to the study’s goals and is able to provide advice and assistance throughout the course of the study. This paper describes studies carried out in three different industrial environments, discusses obstacles that arise, and how the authors have been able to overcome some of those obstacles. 

  • 31. Weyuker, Elaine
    et al.
    Ostrand, Thomas J.
    Software Testing Research and Software Engineering Education2010Conference paper (Refereed)
  • 32. Weyuker, Elaine
    et al.
    Shin, Y
    Ostrand, T
    Bell, R
    Does Calling Structure Information Improve the Accuracy of Fault Prediction?2009Conference paper (Refereed)
1 - 32 of 32
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf