The Art of Test Automation

Test automation has evolved to become a strategic and integral part of the software development process. Most of us start our test automation careers with record and playback. Over time, some of us move to data-driven test automation, but very few of us move towards the core where in the principles of design and development are applied to test automation. Test automation is like developing a system where test cases are requirements. The depth of thinking and planning that goes into test automation before hitting the record button is similar to developing software

Over the last 10 years, I have seen multiple Fortune clients struggle with automation and some of them eventually getting it right. For some of the projects that failed, we had the best test automation resources and a very stable manual testing practice but in spite of all this there was a huge gap between what was dreamt and what actually got realized. Over the period, we realized that the planning process is a key component for successful test automation. In 65% of the projects that failed the planning process and sequence of steps followed were the reasons.

Based on my experience, the automation process is:

  • Why (Purpose)
  • When (Stable Setup and Manual Process)
  • Which (Tool Selection)
  • What (Test Case Selection)
  • How (Design)

We have written a detailed whitepaper “The Art of Test Automation” based on the test automation process above. Through this white paper, we have attempted to outline how to actually go about automating, planning, prioritizing and using better practices to ensure a lesser risk of complete failure in automation projects.

Some of the important test automation questions that this paper attempts to address:

  • Why automation fails in spite of having technical resources?
  • Is there a standard process to be followed for test automation?
  • When to start and stop automation?
  • Test selection criterion

Download “The Art of Test Automation” to read more about the ideal automation process.

Poonam Rathi | Test Consultant | Zen Test Labs

Developer + Tester = 1?

Recently I attended a software testing conference where the main focus of discussion revolved around future of testing and how innovation can be ingrained more in testing!

The speakers included Randy Rice, Michael Bolton and Lee Copeland. When asked about the future of testing, and each one said that it was a too dangerous to predict. They realized that when they were asked this question ten years back, their predictions failed thoroughly J  Quoting Neils Bohr “Prediction is very difficult, especially if it’s about the future”.
A common point said by each was that it would be more complex and besides functional testing, security testing would be a major game.

One of the intriguing points I found was in order to enhance creativity,asking the developer to also play tester. I think it is a recipe for disaster to expect a developer to test his baby.

The reasons why the developer and the tester should be different are:

–          The developer works  on building a particular module but the tester has to think of the integrating that part as well, thus varying  the scope and efficiency expected from the tester and the developer

–          The creative process of a developer works on a structured constructing process, where as a tester proves his creativity by breaking the barriers and rules

–          A developer is entitled only to the code and does not work with the mindset of looking for failures, where as a tester whether access to code or no code works towards digging failures

–          However, when a new Functional Specification Document or a Business Specification Document is acquired, the tester and the developer can start working simultaneously, by adopting Agile more productivity and efficiency is attained over a limited period of time.

If I must combine the role of a tester and a developer, then

A developer can enhance his productivity by unit testing, if there is an error in the code it is easier found by the developer rather than a separate tester who might or might not have the knowledge of that technology/language.He can also build by thinking of end to end business flows.

A tester can wear the developer’s shoes by building automated scripts and applying Oops concepts in his test cases.

I believe this would also add in enhancing the innovation in testing! These are my views acquired over dedicating a period of time in testing, Would be great to know yours?

Poonam Rathi |Test Consultant | ZenTest Labs

Common Issues and Solutions in Selenium

Selenium is fast becoming one of the most widely used test automation tools for web based applications. While working on selenium projects we faced some common issues that I thought is best addressed through the solutions I am listing below. In case you have tried other solutions and they have worked for you, please feel free to add your inputs in the comments box.

Issue No. 1. Issue in handling popup windows / Frame
Solution: Pop-ups that are very commonly found on the GUI of web apps are many times not recorded by selenium or cause issues while running selenium scripts. The table below gives a snapshot at how these issues can be handled

Popup Type Description Solution
Alert Popup (Message box) Message box popup on application To handle alert popup you can use getAlert functions e.g. selenium.GetAlert();
This will click on Ok button of alert and return alert message.
Confirmation Popup This type of popup has Ok & Cancel buttons You can select the button you want to click, by using below statements before GetConfirmation
For Ok – chooseOkOnNextConfirmation
For Cancel – chooseCancelOnNextConfirmation
And then use GetConfirmation as follows,
selenium.GetConfirmation();
This will click on selected button & return confirmation message.
Popup Window/ Frame In this type of popup, the Frame/window contains data fields where we have to fill values in fields. this can be done using SelectFrame
Following code will illustrate how this can be done.

selenium.Click(btnSubmit)
// This will open a popup window asking user to enter activation code & click on confirm button
selenium.SelectFrame(frmActivationCode); // this will transfer the control to Activation code frame i.e. popup window
//Now you can process objects from popup frame
selenium.Type(txtactivationCode, stractivecode);
selenium.Click(btnConfirm);
// After this you have to get back the control to parent window again using
selenium.SelectWindow(“”);

Issue No.2. Selenium does not trigger event on changing values
Solution: Selenium does not trigger event on changing values so we have to explicitly trigger the event using fireEvent,

selenium.FireEvent(cmbCategory, “onchange”);
Here, cmbCategory is object on which you want to fire event & onchange is the name of event.

Issue No.3. Synchronization issue leading to timeout
Solution:
For this type of scenario I created a function ‘CheckElementPresent’ which will take object locator as parameter & will return true once object is visible.

CheckElementPresent(btnSubmit)
Here, I have used selenium.IsElementPresent(locator) to verify object is present in for loop with Thread.Sleep

Issue No.4. Handling SSL certificates
Solution: Many times we face issue in SSL certificates for selenium tests. We can handle this issue by using –trustAllSSLCertificates & firefox profiles as shown below,
a. For browser other than firefox
Use following command to start selenium server, this will start selenium server which trust all SSL certificates.
C: > java -jar c:\Selenium\selenium-server2.14.0.jar. –trustAllSSLCertificates
b. For Firefox browser
Create a new firefox profile (steps to create profile – http://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles)
And accept SSL Certificate using same profile. Now, use the same profile while running selenium server using following command,

C:> java -jar c:\Selenium\selenium-server2.14.0.jar. -firefoxProfileTemplate /C:\Selenium\ Profile

Here were some issues that I faced and the solutions I came up with. Please share any other solutions you know of for the above issues. Also please share other issues that you faced and if possible provide the solutions that you came up with. Happy testing! 

Hemant Jadhav | Senior Consultant | ZenTest Labs

Automating data migration testing

I had the opportunity to be a part of data migration project recently. I was involved in automated data migration testing, which I found it to be a very exciting and challenging form of testing.  I wanted to share my learning’s in this post.

During conversion or migration projects, data from the legacy or source systems is extracted, transformed and loaded into the target system. This is called as the ETL process. The data needs to be transformed as the data model of any two systems is different. As a standard practice the data transformations are managed in the data mapping document, which forms the basis for development as well as testing.

Testing the migrated data is usually a critical and challenging aspect. Testing the migrated data manually is a very time consuming process and so automated data validation is a good way to go ahead.

In my latest project, data from two source systems was migrated to target system. The data from the source systems’ UI was compared with the data from target system’s UI, since we did not have access to the database. Data migration was performed based on incremental loading approach to ensure cent per cent verification. The approach was to load small subsets of data every week for verification. This type of a process was a perfect solution to client’s challenges as in the event of any mismatch only that specific subset of data could be reversed.

I am also listing some of the key challenges we overcame during the course of the project

  1. We had to create scripts that could read source values and the use field level mapping rules to calculate the expected results at the destination. This had to be done because the mappings between the fields of the source system and the target system were different; i.e., both systems had their own structure
  2. We had to verify values at the target system as some extra fields were present in it leading to a mismatch with the source system
  3. We had to read the data on the target system to verify as some amount of data in the source system was in a CSV format with header changes for each customer column
  4. We also created a strong log generation mechanism that generated a result for every iteration. It also went onto ensure that when any mismatch occurs not only field name mismatches are captured but also values get captured
  5. The results also included the time taken to execute each record
  6. To counter the fact that most of the data migration was done in files of XML tab separated formats, we had to generate the input file for automation in excel format

We also went onto to create a customized data migration automation testing framework (illustrated below) to overcome these challenges which lead to a successful project.

Have any of you worked on such projects? Would love to hear some of your experiences.

Anand Gharge | Test Manager | Zen Test Labs