Optimizing Test Design

Boris Beizer, a definitive guru in the world of software testing, famously said “More than the act of testing, the act of designing tests is one of the best bug preventers known.”  Proactive test design can help you build quality into the system instead of testing for defects towards the end of the software development lifecycle. It allows testers to identify and eliminate defects before they are coded.

Test case design is also a powerful tool for risk analysis and optimization since it decides which parts of the system are to be tested. More often than not, the number of test cases created is far greater than the time and resources available to execute them. In such cases, good test design can be used to select the right number of tests that provide optimum coverage. Test design is also a huge factor in the success of any test automation project, even more so than tool selection and scripting.

However, a single test case design technique cannot be adequate to fully test a product. It is advisable to use a combination of these techniques because the value of each technique depends on the context of the functionality being tested. At Zentest, we use a combination of traditional and advanced test case design techniques to produce an optimized set of test cases that provide maximum test coverage. We have outlined these techniques below (click image to enlarge):

Img1

Advanced techniques such as Classification Tree are also immensely helpful in optimizing test cases. Classification Trees identify test relevant aspects which are called classifications and their corresponding values which are referred to as classes. The different classes from all classifications are then combined into test cases. Test design becomes visual through Classification Trees which makes communication and understanding easy. This technique provides the bare minimum test cases in a way that every ‘classification’ is covered at least once.

In the example below, a maximum of 12096 (2 X 8 X 7 X 2 X 2 X 2 X 3 X 3 X 3) test cases are required to test every possible combination. With Classification Trees, a minimum of 8 test cases and an optimal of 128 are enough.

Img2

Designing test cases is as much an art as a science.  At the core of good test design is a combination of test design techniques backed by creative and critical thinking.  The objective of good test design is to create reasonably sized tests that are aggressive enough to find defects. Reducing test design to a dull, mechanical activity will result in shallow and obvious tests that are unable to break the system and find defects.   In our experience, more than a specific technique, a combination of test design techniques backed by creative and critical thinking can lead to optimal results.

Zen Test Labs | ‘Go-Live’ Faster !

The evolution of test design techniques

Test Design is a constantly evolving area of the software testing life cycle. Surprisingly, it still remains implemented at a relatively smaller scale whenever new application features are defined and this has gone a long way in limiting the effectiveness and efficiency of testing.

The heart of it has always been the test design template, the primary goal of which is to reduce ad-hoc reinvention of test designs. However, most companies/ individuals do not assign as much importance to this as they do to the development process. Ideally, a good test design frees tester to focus on the actual job of testing the application. I have seen that just by using multiple test design techniques, the efficiency gain that we have been able to achieve has been tremendous. Some of them include

  • Specification based techniques like equivalence partitioning, boundary value analysis, state transition testing, use case testing and decision table testing
  • Structure based techniques/ White box testing through statement testing & coverage and decision testing coverage
  • Experience based techniques like error guessing and exploratory testing

Our teams have been using these techniques for quite some time now and have come up with innovative ways of creating classification trees, in order to effectively design and generate test cases, for complicated applications.  One way is, wherein we map the required output so that the classification of inputs and outputs is matched.

We are in the process of publishing a whitepaper on this area early next quarter that details out how one should go about Test Design.

Watch this space for more on the paper…

Sunil Deshmukh | Senior Test Manager | Zen Test Labs