mdh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Quality of Test Design in Test Driven Development
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0001-8009-9052
2013 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

One of the most emphasised software testing activities in an Agile environment is the usage of the Test Driven Development (TDD) approach. TDD is a development activity where test cases are created by developers before writing the code, and all for the purpose of guiding the actual development process. In other words, test cases created when following TDD could be considered as a by-product of software development. However, TDD is not fully adopted by the industry, as indicated by respondents from our industrial survey who pointed out that TDD is the most preferred but least practised activity.

Our further research identified seven potentially limiting factors for industrial adoption of TDD, out of which one of the prominent factor was lack of developers’ testing skills. We subsequently defined and categorised appropriate quality attributes which describe the quality of test case design when following TDD. Through a number of empirical studies, we have clearly established the effect of “positive test bias”, where the participants focused mainly on the functionality while generating test cases. In other words, there existed less number of “negative test cases” exercising the system beyond the specified functionality, which is an important requirement for high reliability systems. On an average, in our studies, around 70% of test cases created by the participants were positive while only 30% were negative. However, when measuring defect detecting ability of those sets of test cases, an opposite ratio was observed. Defect detecting ability of negative test cases were above 70% while positive test cases contributed only by 30%.

We propose a TDDHQ concept as an approach for achieving higher quality testing in TDD by using combinations of quality improvement aspects and test design techniques to facilitate consideration of unspecified requirements during the development to a higher extent and thus minimise the impact of potentially inherent positive test bias in TDD. This way developers do not necessarily focus only on verifying functionality, but they can as well increase security, robustness, performance and many other quality improvement aspects for the given software product. An additional empirical study, evaluating this method, showed a noticeable improvement in the quality of test cases created by developers utilising TDDHQ concept. Our research findings are expected to pave way for further enhancements to the way of performing TDD, eventually resulting in better adoption of it by the industry.

Place, publisher, year, edition, pages
Västerås: Mälardalen University , 2013. , 224 p.
Series
Mälardalen University Press Dissertations, ISSN 1651-4238 ; 140
Keyword [en]
agile, software testing, test driven development, quality of tests, developer testing, test design quality
National Category
Computer Science
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:mdh:diva-18773ISBN: 978-91-7485-108-3 (print)OAI: oai:DiVA.org:mdh-18773DiVA: diva2:618378
Public defence
2013-06-11, Gamma, Högskoleplan 1, Västerås, 09:15 (English)
Opponent
Supervisors
Available from: 2013-04-29 Created: 2013-04-26 Last updated: 2015-11-04Bibliographically approved
List of papers
1. An Industrial Survey on Contemporary Aspects of Software Testing
Open this publication in new window or tab >>An Industrial Survey on Contemporary Aspects of Software Testing
2010 (English)In: ICST 2010 - 3rd International Conference on Software Testing, Verification and Validation, Paris, France, 2010, 393-401 p.Conference paper, Published paper (Refereed)
Abstract [en]

Software testing is a major source of expense in software projects and a proper testing process is a critical ingredient in the cost-efficient development of high-quality software. Contemporary aspects, such as the introduction of a more lightweight process, trends towards distributed development, and the rapid increase of software in embedded and safety-critical systems, challenge the testing process in unexpected manners. To our knowledge, there are very few studies focusing on these aspects in relation to testing as perceived by different contributors in the software development process. This paper qualitatively and quantitatively analyses data from an industrial questionnaire survey, with a focus on current practices and preferences on contemporary aspects of software testing. Specifically, the analysis focuses on perceptions of the software testing process in different categories of respondents. Categorization of respondents is based on safety-criticality, agility, distribution of development, and application domain. While confirming some of the commonly acknowledged facts, our findings also reveal notable discrepancies between preferred and actual testing practices. We believe continued research efforts are essential to provide guidelines in the adaptation of the testing process to take care of these discrepancies, thus improving the quality and efficiency of the software development.

Place, publisher, year, edition, pages
Paris, France: , 2010
Identifiers
urn:nbn:se:mdh:diva-9339 (URN)10.1109/ICST.2010.52 (DOI)2-s2.0-77954480832 (Scopus ID)978-076953990-4 (ISBN)
Conference
3rd International Conference on Software Testing, Verification and Validation, ICST 2010; Paris; 7 April 2010 through 9 April 2010
Available from: 2010-03-03 Created: 2010-03-03 Last updated: 2013-12-03Bibliographically approved
2. Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review
Open this publication in new window or tab >>Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review
2011 (English)In: 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011; Berlin; 21 March 2011 through 25 March 2011, 2011, 337-346 p.Conference paper, Published paper (Refereed)
Abstract [en]

Test driven development (TDD) is one of the basic practices of agile software development and both academia and practitioners claim that TDD, to a certain extent, improves the quality of the code produced by developers. However, recent results suggest that this practice is not followed to the extent preferred by industry. In order to pinpoint specific obstacles limiting its industrial adoption we have conducted a systematic literature review on empirical studies explicitly focusing on TDD as well as indirectly addressing TDD. Our review has identified seven limiting factors viz., increased development time, insufficient TDD experience/knowledge, lack of upfront design, domain and tool specific issues, lack of developer skill in writing test cases, insufficient adherence to TDD protocol, and legacy code. The results of this study is of special importance to the testing community, since it outlines the direction for further detailed scientific investigations as well as highlights the requirement of guidelines to overcome these limiting factors for successful industrial adoption of TDD.

National Category
Computer and Information Science
Identifiers
urn:nbn:se:mdh:diva-12206 (URN)10.1109/ICST.2011.19 (DOI)2-s2.0-79958704204 (Scopus ID)978-076954342-0 (ISBN)
Conference
Fourth International Conference on Software Testing, Verification and Validation
Available from: 2011-05-10 Created: 2011-05-06 Last updated: 2013-12-03Bibliographically approved
3. Impact of Test Design Technique Knowledge on Test Driven Development: A Controlled Experiment
Open this publication in new window or tab >>Impact of Test Design Technique Knowledge on Test Driven Development: A Controlled Experiment
2012 (English)In: Lecture Notes in Business Information Processing, vol. 111 LNBIP, Springer, 2012, 138-152 p.Chapter in book (Refereed)
Abstract [en]

Agile development approaches are increasingly being followed and favored by the industry. Test Driven Development (TDD) is a key agile practice and recent research results suggest that the successful adoption of TDD depends on different limiting factors, one of them being insufficient developer testing skills. The goal of this paper is to investigate if developers who are educated on general testing knowledge will be able to utilize TDD more efectively. We conducted a controlled experiment with master students during the course on Software Verification & Validation (V&V) where source code and test cases created by each participant during the labs as well as their answers on a survey questionnaire were collected and analyzed. Descriptive statistics indicate improvements in statement coverage. However, no statistically significant differences could be established between the pre- and post-course groups of students. By qualitative analysis of students’ tests, we noticed a lack of test cases for non-stated requirements ("negative" tests) resulting in a non-detection of bugs. Students did show preference towards TDD in surveys. Although further research is required to fully establish this, we believe that identifying specific testing knowledge which is complementary to the testing skills of a new TDD developer would enable developers to perform their tasks in a more eficient manner.

Place, publisher, year, edition, pages
Springer, 2012
National Category
Engineering and Technology
Identifiers
urn:nbn:se:mdh:diva-17243 (URN)10.1007/978-3-642-30350-0_10 (DOI)2-s2.0-84862096524 (Scopus ID)978-364230349-4 (ISBN)
Note

13th International Conference on Agile Software Development, XP 2012; Malmö; 21 May 2012 through 25 May 2012

Available from: 2012-12-20 Created: 2012-12-20 Last updated: 2016-05-17Bibliographically approved
4. Quality of Testing in Test Driven Development
Open this publication in new window or tab >>Quality of Testing in Test Driven Development
2012 (English)In: 2012 8th International Conference on the Quality of Information and Communications Technology, QUATIC 2012: Proceedings, IEEE , 2012, 266-271 p.Conference paper, Published paper (Refereed)
Abstract [en]

Test-driven development is an essential part of eXtreme Programming approach with the preference of being followed in other Agile methods as well. For several years, researchers are performing empirical investigations to evaluate quality improvements in the resulting code when test-driven development is being used. However, very little had been reported into investigating the quality of the testing performed in conjunction with test-driven development. In this paper we present results from an experiment specifically designed to evaluate the quality of test cases created by developers who used the test-first and the traditional test-last approaches. On an average, the quality of testing in test-driven development was almost the same as the quality of testing using test-last approach. However, detailed analysis of test cases, created by test-driven development group, revealed that 29% of test cases were "negative" test cases (based on non-specified requirements) but contributing as much as 65% to the overall tests quality score of test-first developers. We are currently investigating the possibility of extending test-driven development to facilitate non-specified requirements to a higher extent and thus minimise the impact of a potentially inherent effect of positive test bias.

Place, publisher, year, edition, pages
IEEE, 2012
National Category
Engineering and Technology
Identifiers
urn:nbn:se:mdh:diva-17339 (URN)10.1109/QUATIC.2012.49 (DOI)000319880600043 ()2-s2.0-84878829241 (Scopus ID)978-076954777-0 (ISBN)
Conference
8th International Conference on the Quality of Information and Communications Technology (QUATIC), Lisbon, Portugal, 3-6 September 2012
Available from: 2012-12-20 Created: 2012-12-20 Last updated: 2013-12-03Bibliographically approved

Open Access in DiVA

kappa(378 kB)595 downloads
File information
File name FULLTEXT01.pdfFile size 378 kBChecksum SHA-512
304909ebf8ae75fe84e55129fb704b080065f1fa0e4774e31035c5e52f611b90495ebec056f0ece41e70b7ae3065386825af410c127b46aed21117e5de8dae29
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Čaušević, Adnan
By organisation
Embedded Systems
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 595 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 1773 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf