8 Steps to Improve Your Regression Testing Process

With business and user requirements perpetually in an evolutionary mode, I find that regression testing has become a key component of the software development lifecycle. As testers, we need to keep in mind that a constant change in the functionality of the application lends the system to vulnerabilities in the base functionality too. These vulnerabilities tend to creep in due to an oversight while adding new functionality, poor analysis of impact on interfacing/ integrating applications and many a times due to the fact that customizations are an unknown entity. Poor regression testing can not only result in poor software quality but also impact revenue and cause customer loss.
Based on many years of planning, creating and executing the Quality Assurance programs of multiple Fortune 500 companies, I suggest the following eight step methodology to improve any regression testing process.

Phase 1: Defining
Step 1: Objective Finding (OF) – Challenges and Goal Identification
This step answers one of the most important questions “Why is regression testing not effective in its current state?”

Step 2: Fact Finding (FF) – Data Collation and Analysis
During this stage, teams must trail defects found in the past to conduct a defect root cause analysis. An important part of this step is bug prediction analysis so that defect prone areas in the application can be found.

Step 3: Problem Finding (PF) – Problem Clarification and Statement
Once the results of Steps 1 and 2 are combined, the exact scope of the challenges to address is established. These refined objectives act as the equivalent of a “Requirements Document”.

Phase 2- Scoping
Step 4: Test Cases Finding (TF) –Coverage Gap Analysis
Gaps in test coverage are found based on the current test cases and the application functionality. Techniques to map test cases to requirements and testing techniques are used to identify missing test cases

Step 5: Test Case Centralization (TC) – Test Case Repository Creation
Ensure that all test cases are stored in a centralized repository and in an optimized manner. Each test case must have a clear objective, precondition, steps, expected result and test data.

Step 6 : Test Case Optimization (TO) – Maximum Coverage in Desired Time with Minimum Risk
Statistical techniques such as Classification Tree and Orthogonal Array may be used to run minimum number of test cases in a way where every business process/ function is covered at least once

Phase 3- Executing
Step 7: Reusing Test Components (RT) – A Modular Approach
Create business functions and test data in a way that they can be reused for building manual test cases. Automate the generation of descriptive manual test cases.

Step 8: Test Case Classification (TC) – Test Case Mapping
At this stage, test cases are grouped requirement wise, screen wise, module wise, etc. Small frequently used regression pack/suites are created.

We have written a detailed whitepaper ‘Progress Not Regress’ on improving any regression testing process. We would love to hear your thoughts on it!

Girish Nair | Sr. Consultant | Zen Test Labs

Why automating regression cycles is a no brainer?

It is very surprising when you talk to customers who find regression cycles either painful or a drain on their productivity and at times a distraction from critical projects. What comes as a surprise is that most of them have an extremely mature and stable application and testing process. Yet, around regression cycles they scamper to get the entire suite executed and face the dilemma of having to deploy before completing the cycle.

Manual regression testing can be a mundane and boring activity for most test teams! Going through screens repeating the same steps and not finding bugs puts a tester’s concentration levels to the test. It also risks critical bugs escaping through due to the way these suites are structured. Over the years of executing numerous regression cycles I have seen that while automating regression is a great way to address the issue, other factors need to be considered to make regression cycles a cakewalk. I am outlining some of the steps that my teams find useful while automating regression suites

  • Find out how many of the existing test cases are automatable
  • Find out the coverage gap to identify ideal number of test cases
  • Run a test optimization drive (based on scientific methods) to arrive at the optimum number of test cases while ensure maximum coverage
  • Do not jump into QTP (explore other low cost options including Selenium)
  • Use a test automation framework that enables business users to develop, manage and execute automated regression suites
  • Train business users to automate on the fly and use the automation engineer only to maintain.

I acknowledge and understand that one can never achieve 100% test automation but the way I look at it, if you can achieve and significant amount (70-80%) you are going to end up achieving a dramatic reduction in the time taken to execute regression cycles. Eliminating pure testers from this process is also truly empowering your test teams to focus on critical projects.

Hari Raghunathan | AVP | Zen Test Labs