Communication- A Key to Excel at Testing

Enabling better communication is not a onetime activity. It requires continuous effort across the company. Good internal and external communication is extremely important to a business’s success. In order to work together effectively, there must be clear and coherent communication among all the departments.

Here are a few scenarios wherein communication gaps may arise and lead to poor quality:

1. Continuously Changing Requirements:
At times, requirement changes are implemented directly without updating the specification document. In such cases, there is a chance that the changed requirements remain untested or are tested incorrectly.
Any change in the requirements should be communicated correctly to all stake holders and it is necessary to update the specification document on a timely basis.

2. Configurations:
Lack of clarity from the stakeholders on the configurations to be tested can lead to wasted effort and extra work. Configuration testing can be expensive and time consuming. Investment in hardware and software is required to test the different permutations and combinations. There is also the cost of test execution, test reporting and managing the infrastructure.
Increasing communication between development, QA and stakeholders can help deal with these challenges.

3. Team Size:
When team sizes are large, some members of the team may miss changes in requirements or may not be communicated updates on activities in the project. This could lead to severe problems in the project or project failure.  Each team member should be abreast of the activities in the project through a log or through other means.

4. Changes in  Application Behavior not Communicated:
Continuous changes in the application behavior may lead to requirements being tested incorrectly.  All the functionality implemented in the application should be frozen while testing. If any changes are made to the functionality, they should be communicated to the testing team on a timely basis.

5. Unclear Requirements:
Complex requirements that contain insufficient data may be difficult to understand and therefore, may lead to improper testing. The functional/technical specification documents should be clear and easy to understand; they should contain a glossary, screenshots and examples wherever necessary.

The path to project success is through ensuring that small communication problems are eliminated completely before they build up, so that the message is delivered correctly and completely. Instead of discovering problems, we should figure out how to stop them from appearing in the first place.

Poonam Rathi | Test Consultant | Zen Test Labs

Major Checkpoints for Database Migration Testing

I had the opportunity to work on my very first data migration project a few days ago. The objective of our project was to ensure the correctness and completeness of data migrated from a source system with Oracle as its database to a target system with MS SQL as its database.  I learned a lot while working on this project. I got an opportunity to hone my SQL query writing skills and got well versed with the major checkpoints and activities performed during database migration testing. I wanted to share my learning’s and experience in this post.

23 million data sets had to be migrated from Oracle to SQL. In order to do this, we divided the activities into two phases:

Info1Our job was to test and verify the correctness and completeness of around 23 million data sets at the end of each activity. The real challenge was to perform this job manually without automation. What made it even more interesting was that there’s a chance of up to 5% data loss during data migration.

To achieve our goal, we divided our tasks into 4 major Phases:

Info2

The experience I gained while working on this project can be divided into 3 parts:

  1. Basic skills required for database migration testing
  2. Major checkpoints for database migration testing
  3. Major activities performed during database testing

Basic skills required for database migration testing

Other than testing skills required for all types of testing, one needs following skills for data migration testing

  1. SQL query writing for legacy and target database
  2. Knowledge of the data migration testing tool (if any) being used or advance knowledge of Excel features for data comparison

Major checkpoints for database migration testing, data profile matching

  1. Data profiling
  • Table count
  • Data type matching of columns
  • Identifying different classes of records from business point of view.
  • Performing Check Sum on column holding numerical data
  • Row count matching of legacy and target database
  • Column count matching of legacy and target database
  • Matching of total Null values per column of legacy and target database
  • Matching of Not Null count per column
  • Checking the Distinct count per column
  • Checking the Group By count per column
  • Matching of Summation of numeric fields
  • Checking the Min value per numeric column
  • Checking of Max value per numeric column
  • Matching of Average data value in numeric columns
  • Checking the biggest values in non-numeric columns
  • Checking the shortest values in non-numeric columns

2. Checking  data redundancy
3. Matching  control tables to verify the exact transfer of relationships among tables
4. Functional testing on migrated data
5. Random sampling- Picking and matching data from corresponding tables randomly.

Major activities performed during database testing

  1. Identification and matching of the number of tables in legacy and target databases.
  2. Identification and matching of the data types of columns in corresponding tables.
  3. Identification and matching of the relationship between tables in legacy database and target database.
  4. Identification and matching of business rules in legacy database.
  5. Identification of primary key attribute for each table
  6. Identification of columns with Not Null property
  7. Identification of numeric fields for data profiling
  8. Query writing for both databases to identify all data profiling checkpoints:
    1. Row count
    2. Column count
    3. Null values count per column
    4. Not Null count per column
    5. Distinct values count per column
    6. Group By count per column
    7. Summation of numeric fields
    8. Min value per numeric column
    9. Max value per numeric column
    10. Average value of numeric fields
    11. Biggest values in non-numeric columns
    12. Shortest values in non-numeric columns
  9. Matching the data profiling results of both databases
  10. Query writing for random sampling to ensure that data has been migrated correctly and completely
  11. Using migrated data with the related application to test its functional use

Migrating data is a challenging activity. Users expect it to be fast and efficient, without loss of data.  Thorough data migration testing has to be performed to reduce risk and ensure that data has been migrated as per requirements. I believe that using the points outlined above, you can refine your test approach and ensure a better data migration testing process.

Mayank Raj | Trainee Test Analyst | Zen Test Labs

The Art of Test Automation

Test automation has evolved to become a strategic and integral part of the software development process. Most of us start our test automation careers with record and playback. Over time, some of us move to data-driven test automation, but very few of us move towards the core where in the principles of design and development are applied to test automation. Test automation is like developing a system where test cases are requirements. The depth of thinking and planning that goes into test automation before hitting the record button is similar to developing software

Over the last 10 years, I have seen multiple Fortune clients struggle with automation and some of them eventually getting it right. For some of the projects that failed, we had the best test automation resources and a very stable manual testing practice but in spite of all this there was a huge gap between what was dreamt and what actually got realized. Over the period, we realized that the planning process is a key component for successful test automation. In 65% of the projects that failed the planning process and sequence of steps followed were the reasons.

Based on my experience, the automation process is:

  • Why (Purpose)
  • When (Stable Setup and Manual Process)
  • Which (Tool Selection)
  • What (Test Case Selection)
  • How (Design)

We have written a detailed whitepaper “The Art of Test Automation” based on the test automation process above. Through this white paper, we have attempted to outline how to actually go about automating, planning, prioritizing and using better practices to ensure a lesser risk of complete failure in automation projects.

Some of the important test automation questions that this paper attempts to address:

  • Why automation fails in spite of having technical resources?
  • Is there a standard process to be followed for test automation?
  • When to start and stop automation?
  • Test selection criterion

Download “The Art of Test Automation” to read more about the ideal automation process.

Poonam Rathi | Test Consultant | Zen Test Labs

Using Mind Maps in Testing

Mind maps are an excellent tool and can be used in a variety of testing activities like requirements analysis, test design, test planning, session reports, measuring test coverage etc. Testing relies heavily on communicating stories about what should be tested, how it should be tested, what are the risky areas and so on. Making this process visual can help testing teams articulate their thoughts & ideas better. Drawing mind maps also makes generating new ideas much easier.

Take a look at the simple “Replace” dialogue box below

Dialogue Box

We can easily create a mind map for testing this functionality using the following steps:

  1. Draw the main theme in the centre.
  2. Draw the module name/features of the application branching out from the main theme.
  3. Draw the sub module/feature branching out from each module/feature.
  4. Add colors to your mind map to make it easier for your brain to group things together.
  5. Write test cases for each feature and sub feature.
  6. Include only testable items in your mind map
  7. Try not to use full sentences in your mind map

Mind Map 1

Some examples for creating exclusive mind maps or creating branches in existing mind maps are:

  • Mind maps for field level validation of all fields on the screen
  • Identify fields that are common to all screens and create a ‘Common Fields’ mind map. Eg. Date Field- this field is the same in all screens
  • Mind maps that include business rules
  • Mind maps for events like Mouse Over, Click etc
  • Mind maps based on Screen Names
  • Mind maps based on Functionality

An example Mind Map for validating a subscription form

Subscription Form Mind Map

Ideas for using mind maps in testing:

  • Mind Map Jamming: All the testing team members read /analyze a particular requirement/feature and create a mind map for it together.
  • Using Mind Maps for Defect/ Execution Summary: Create a mind map of test cases. After execution, you can mark (tick or cross) the mind map as per the Actual Result, thus using it to provide Defect/Execution Summary.
  •  Smoke/Sanity Testing: Create a mind map for all the flows that are to be Smoke tested or Sanity tested.
  • Scope: Create a mind map to show what is in Scope and what is not in Scope.

You can use mind maps anywhere and everywhere! Mind maps exist to make your life easy, so if a mind map is getting too big or complicated try splitting it.  The great thing about mind maps is that all test cases are visible in one view; you don’t need to scroll up and down. This also makes it simpler to add new points whenever you want. Mind maps provide more coverage and the likelihood of missing important points is lesser. You cannot use long detailed sentences in mind maps. Using one word per line improves clarity and understanding. It makes recollection easier. Using single keywords will make your mind maps more powerful and flexible.

 Mindmap testing-final

Mind mapping skills improve over time and with practice your mind maps will become more extensive & wide-ranging. Although, mind maps help you simplify information and make it easily understandable, you must not forget that they are ultimately models and therefore, they may leave out important aspects. So make sure that you question what might be missing from the map and add those things. This is quite simple as all you have to do is add another node to the map!

Satish Tilokchandani | Lead Consultant | Zen Test Labs

Pressure to Release and the Impact on Testing

The pressure to release new products to a consumer with the appetite of a blue whale has led to an unprecedented situation. Companies are being pushed to embrace iterative and incremental development methodologies in a bid to keep users happy. Add what is now commonly known as the Google effect “I want it now and I want it free” and you have a disaster of epic proportions in the making.

I’d like to draw a parallel from the aircraft manufacturer industry.Airbus and Boeing have been in a battle for over 20 years now. This battle reached a crescendo in 2005 with the launch of the world’s largest passenger aircraft; i.e., the Airbus 380 releasing an extremely high pressure situation on Boeing to launch a new aircraft.Boeing worked on trying to get a better aeroplane that could beat the A380. Boeing finally delivered on its promise in 2007 with the launch of its own 787 Dreamliner. Unfortunately, Boeing has been plagued with a host of issues on the 787s including electrical fires on board mid-air, leading to the grounding of the entire fleet. Fortunately, there have been no fatal accidents to date.

Why do I state this example in a software testing blog?

The aircraft industry is a cutting edge world with great emphasis on passenger safety, deep rigor in testing all equipment thoroughly before allowing actual passengers to fly. I am very confident that Boeing would have followed a meticulous protocol to test this aircraft but somewhere the pressure to release has led to errors creeping in the process. Boeing’s issues reinforces the fact that we live in a world where the pressure to release can get to the best of us.Now this is an industry where testing is not taken lightly and this happened. Over the years, the kind of trade-offs I have seen software companies make I am surprised most of them have lasted this long :)

Given all these factors, we all acknowledge and accept that we cannot possibly test every point in a system to find defects. Thus one of the ways that has evolved of late is to run a “Risk based testing” program that protects you from critical failure in projects

Most customers I interact with end up using models that work on probabilities, impact, time and cost. One recommendation we end up making to most of them is to use statistical models to do this analysis. Do not rely only on past data, experience, user feelings, etc. Put it in a statistical model and see what the scenarios that come out are. Map these to your past data, experience, high failure areas, criticality to business, user feeling, etc. and add/ edit/ delete based on that. What you will end up with is a decent test plan that covers your risk sufficiently. It is critical that you revisit this periodically to adjust your plan on defects you are finding.

Ultimately, the decision to release a product is one driven by business needs. However, shipping a product that is not tested properly may not be that great an idea as a small number of people can make a lot of noise about problems-even those problems that are not so serious. Unfortunately that’s the world we live in now! As for Boeing, there is always the next battle to win in a duopoly .

Hari Raghunathan | AVP | Zen Test Labs

8 Steps to Improve Your Regression Testing Process

With business and user requirements perpetually in an evolutionary mode, I find that regression testing has become a key component of the software development lifecycle. As testers, we need to keep in mind that a constant change in the functionality of the application lends the system to vulnerabilities in the base functionality too. These vulnerabilities tend to creep in due to an oversight while adding new functionality, poor analysis of impact on interfacing/ integrating applications and many a times due to the fact that customizations are an unknown entity. Poor regression testing can not only result in poor software quality but also impact revenue and cause customer loss.
Based on many years of planning, creating and executing the Quality Assurance programs of multiple Fortune 500 companies, I suggest the following eight step methodology to improve any regression testing process.

Phase 1: Defining
Step 1: Objective Finding (OF) – Challenges and Goal Identification
This step answers one of the most important questions “Why is regression testing not effective in its current state?”

Step 2: Fact Finding (FF) – Data Collation and Analysis
During this stage, teams must trail defects found in the past to conduct a defect root cause analysis. An important part of this step is bug prediction analysis so that defect prone areas in the application can be found.

Step 3: Problem Finding (PF) – Problem Clarification and Statement
Once the results of Steps 1 and 2 are combined, the exact scope of the challenges to address is established. These refined objectives act as the equivalent of a “Requirements Document”.

Phase 2- Scoping
Step 4: Test Cases Finding (TF) –Coverage Gap Analysis
Gaps in test coverage are found based on the current test cases and the application functionality. Techniques to map test cases to requirements and testing techniques are used to identify missing test cases

Step 5: Test Case Centralization (TC) – Test Case Repository Creation
Ensure that all test cases are stored in a centralized repository and in an optimized manner. Each test case must have a clear objective, precondition, steps, expected result and test data.

Step 6 : Test Case Optimization (TO) – Maximum Coverage in Desired Time with Minimum Risk
Statistical techniques such as Classification Tree and Orthogonal Array may be used to run minimum number of test cases in a way where every business process/ function is covered at least once

Phase 3- Executing
Step 7: Reusing Test Components (RT) – A Modular Approach
Create business functions and test data in a way that they can be reused for building manual test cases. Automate the generation of descriptive manual test cases.

Step 8: Test Case Classification (TC) – Test Case Mapping
At this stage, test cases are grouped requirement wise, screen wise, module wise, etc. Small frequently used regression pack/suites are created.

We have written a detailed whitepaper ‘Progress Not Regress’ on improving any regression testing process. We would love to hear your thoughts on it!

Girish Nair | Sr. Consultant | Zen Test Labs

The chronicles of a new tester

The general mentality in the software testing industry is ‘Negative thinking is one of the most desired attributes of a software tester’. Ever since my sophomore days I aspired to be a software tester. I wondered if being an optimist would hamper my chances of success in the testing field. Many questions stormed my mind. Some of them were:

1. Does this field belong only to negative thinkers?

2. Is negativity the first criteria to become a software tester?

3. What will be the nature of the teams I work with?

4. Is this profession going to change my attitude?

5. If yes, then what kind of a life am I going to lead?

A lot of times I felt like I was passionate about a profession that did not suit me. Keeping all my apprehensions aside, I continued working towards my goal and left no stone unturned in becoming the tester I dreamt to be.

When I practiced test case writing I would come up with at least 10 negative test cases against 1 or 2 positive test cases. For example, I wrote 30 negative test cases and just 2 positive test cases for a simple ‘Change Password’ scenario. This ratio of 1:15 further increased my ‘positive – negative approach’ dilemma. Eventually, I started to believe that I needed to be more of a negative thinker than a positive thinker.

Determination got me into this field.  I have completed 6 months in an independent testing firm and this period has changed my approach towards testing. Working with vastly experienced people in a positive work environment has answered all my queries.

My new approach is

1. Testing doesn’t belong to negative thinkers at all. It belongs to people who can think along multiple directions.

2. Only people with a positive approach can survive in this field. There is no space for negativity.

3. This profession definitely impacts your behaviour outside office; it enables you to think in a 100 different ways about any situation. You can predict 100 different outcomes of an action or incident. You can come up with a wide range of solutions to any problem. So even if there are changes, they are all good.

When I look back, I wonder, where I went wrong. What made me think that? What caught me off guard?

The answer is -wrong terminology-I got stuck in a game of words!

It is not about 2 positive and 30 negative test cases; it is about 2 valid and 30 invalid test cases I brainstormed in 32 creative ways.

I believe, that the terminology ‘positive’ and ‘negative’ test cases should be refrained since they have a tendency to affect the psychology of testing in a negative way.

For me, in the battle of positive and negative thinking, the winner will always be positive thinking, creative thinking!

Mayank Raj | Trainee Test Analyst | Zen Test Labs