Test Automation is Not the Only Answer

I have worked on several test automation projects over the past few years. I also conduct test automation trainings as a part of our company’s CSR initiative and actively participate in online discussions about testing.  Since my work is centered on test automation, a lot of people frequently come to me with questions, some of which I would like to address in this blog post.

Question: “I recently graduated with a Bachelor’s in Computer Science. I am interested in pursuing software testing as a career. Can you recommend an automation tool that I can take up?”

Question: I am interested in software testing and have 3 years of experience in the BPO industry. I also underwent a 3 month QTP and Selenium training. Will this help me get a job as a tester?”

Question: “I have been working as a tester for the last 6 months. I want some growth in my career, so I am planning to move towards test automation. Which tools do you recommend I should learn?”

Question: “I have been working in manual software testing for over 4 years now. I think this has been a great mistake as far as my career is concerned. Most of my colleagues and friends are in test automation and it drives me up the wall .I also want to shift to automated testing; can you guide me as to how I should start?”

These and many similar questions have been asked frequently. All of these questions have a common line of thought: Test Automation. It makes me wonder if automated testing really is more important than manual testing!

Most testers I spoke to wanted to learn test automation only for the following reasons:

  • Knowledge of test automation tools can help their testing career and get them better job opportunities.
  • Some of them wanted to learn test automation just because their colleague was learning it!
  • Adding an extra point in their resume to make it stronger.
  • Highlight the fact that they learned automated testing in their performance appraisal meeting! 🙂

I am not against these testers but want them to realize that automated testing is not the only choice they can make to advance their testing career. Manual testing also offers a lot of growth. Knowledge of automated testing is definitely beneficial, but manual testing is also a very lucrative career path to pursue.

What I’m trying to say is that each of these roles – Manual Testing and Automated Testing have their own very unique challenges. Someone well versed with one role might not necessarily be well-acquainted with another. Treat yourself as a tester; not a manual or automation tester. Think of yourself as a tester with a set of skills, specialties, abilities and domain expertise.

Assuming that automated testing can replace manual testing and using automation tools without understanding testing & the underlying application can be very dangerous. Manual testing is not simple. It’s an art and requires high intelligence, creativity, judgment and skill with domain knowledge.

Finally, remember that human brains cannot be replaced by automated robots. 🙂

Any comments and suggestions are welcome.

Hemant Jadhav | Zen Test Labs

The Art of Test Automation

Test automation has evolved to become a strategic and integral part of the software development process. Most of us start our test automation careers with record and playback. Over time, some of us move to data-driven test automation, but very few of us move towards the core where in the principles of design and development are applied to test automation. Test automation is like developing a system where test cases are requirements. The depth of thinking and planning that goes into test automation before hitting the record button is similar to developing software

Over the last 10 years, I have seen multiple Fortune clients struggle with automation and some of them eventually getting it right. For some of the projects that failed, we had the best test automation resources and a very stable manual testing practice but in spite of all this there was a huge gap between what was dreamt and what actually got realized. Over the period, we realized that the planning process is a key component for successful test automation. In 65% of the projects that failed the planning process and sequence of steps followed were the reasons.

Based on my experience, the automation process is:

  • Why (Purpose)
  • When (Stable Setup and Manual Process)
  • Which (Tool Selection)
  • What (Test Case Selection)
  • How (Design)

We have written a detailed whitepaper “The Art of Test Automation” based on the test automation process above. Through this white paper, we have attempted to outline how to actually go about automating, planning, prioritizing and using better practices to ensure a lesser risk of complete failure in automation projects.

Some of the important test automation questions that this paper attempts to address:

  • Why automation fails in spite of having technical resources?
  • Is there a standard process to be followed for test automation?
  • When to start and stop automation?
  • Test selection criterion

Download “The Art of Test Automation” to read more about the ideal automation process.

Poonam Rathi | Test Consultant | Zen Test Labs

Software Testing in 2020

As a CEO of a testing company, a question that plays on my mind constantly is ‘what is the future of Testing?’ In the early 2000s, Ron Radice spoke at a QAI conference in India, where he had predicted that testing will die. His call was that automatic code generators will do the job so efficiently that testing will become obsolete. When he looked at the crystal ball then, he could see that prevention will be the creed and not detection.

Well, when I look at 2020, I believe Ron was right as well as wrong. Yes, code generators are arriving. Yes, there will be automated test case generators. Yes, model based testing will replace rudimentary testing activities. But, the whole boom of software especially in ubiquitous mobile devices means only more testing.

If the future includes automated cars like the Google driverless cars, I cannot imagine such a car with a technology that has not been fully and manually validated. If the future is the “Internet of Things”, I can only imagine that the amount of embedded testing will only explode. If the future is, business operations being handled through apps and app stores that have millions of applications pervading every step of our business and personal life then imagine the amount of mobile testing that will be required. If not anything, as everything gets more interconnected, the consequences of a critical failure will only be catastrophic. Wherever the nexus of cloud, social, mobile and big data takes us, I am thoroughly convinced that the need for testing will only grow.

While there a dime a dozen predictions on how things will look in 2020, my two bits around where testing will find itself as follows:

• Huge business opportunities arising from testing for app stores directly than app manufactures
• Test automation would have evolved from script less automation to automatic test case generators and execution
• The pressure to deploy rapidly in the Mobility and embedded devices space will mean that test automation tools will evolve to provide near and real time support to these areas
• Testing and testers will evolve to become super specialized with domain testers at one end and niche technical testers at the other end.

These are some things that come to mind and as the decade continues to evolve. Would be great to know what the rest of the testing world thinks.

Krishna Iyer|CEO|Zen Test Labs

Automation lessons learnt: Funding automation projects & the role of change management

One of the many reasons test automation is often compromised is in situations where business funds technology projects on a per project basis. Key reason being business benefits of automation, primarily time to market, are realized only during the subsequent releases of the application, never in the release where automation is undertaken. Even when business agrees to fund an automation project, the order of magnitude of benefits is small due to the potentially low levels of automation feasible within the project scope. The benefits accumulate only over a period of time from increasing automation levels and therefore the return on investment is realized over a longer duration. In order to reap benefits from automation business needs to continually invest in it and maintain a long term orientation to ROI. These are typical characteristics of any change initiatives. Test automation initiatives funded by individual business units can therefore learn from the vast expanse of knowledge pertaining to other organizational change initiatives and do not need to reinvent the wheel.

I have captured my experience of change initiatives as applied to automation in the visual below. It shows key components required to not only succeed at a pilot project but also create a cascading positive spiral where the benefits accumulate over time.

 

Like any other change initiative the key components form a chain, where the initiative is just as strong as its weakest link. Successful pilot accompanied by the right communication can act as a feeder to the next project and as long as all key components act in unison incremental benefits from each project can lead to significant cumulative benefits. The problem is that the first cycle tends to be demanding and it needs continuity of the champions until such time that the framework is institutionalized. A failure at early stages can have devastating effects with a stigma associated with it presenting greater roadblocks during subsequent attempts at automation. This is where senior management support from business and IT is crucial. A champion driving each automation cycle to success is central for the overall success of automation!

What do you think of the role of change management in automation projects? Have you had difficulty funding automation projects? Please feel free to share your experiences.

Aparna Katre | Director Strategy | Zen Test Labs

Automating data migration testing

I had the opportunity to be a part of data migration project recently. I was involved in automated data migration testing, which I found it to be a very exciting and challenging form of testing.  I wanted to share my learning’s in this post.

During conversion or migration projects, data from the legacy or source systems is extracted, transformed and loaded into the target system. This is called as the ETL process. The data needs to be transformed as the data model of any two systems is different. As a standard practice the data transformations are managed in the data mapping document, which forms the basis for development as well as testing.

Testing the migrated data is usually a critical and challenging aspect. Testing the migrated data manually is a very time consuming process and so automated data validation is a good way to go ahead.

In my latest project, data from two source systems was migrated to target system. The data from the source systems’ UI was compared with the data from target system’s UI, since we did not have access to the database. Data migration was performed based on incremental loading approach to ensure cent per cent verification. The approach was to load small subsets of data every week for verification. This type of a process was a perfect solution to client’s challenges as in the event of any mismatch only that specific subset of data could be reversed.

I am also listing some of the key challenges we overcame during the course of the project

  1. We had to create scripts that could read source values and the use field level mapping rules to calculate the expected results at the destination. This had to be done because the mappings between the fields of the source system and the target system were different; i.e., both systems had their own structure
  2. We had to verify values at the target system as some extra fields were present in it leading to a mismatch with the source system
  3. We had to read the data on the target system to verify as some amount of data in the source system was in a CSV format with header changes for each customer column
  4. We also created a strong log generation mechanism that generated a result for every iteration. It also went onto ensure that when any mismatch occurs not only field name mismatches are captured but also values get captured
  5. The results also included the time taken to execute each record
  6. To counter the fact that most of the data migration was done in files of XML tab separated formats, we had to generate the input file for automation in excel format

We also went onto to create a customized data migration automation testing framework (illustrated below) to overcome these challenges which lead to a successful project.

Have any of you worked on such projects? Would love to hear some of your experiences.

Anand Gharge | Test Manager | Zen Test Labs

Implementing Object Repository in Selenium

Selenium (http://seleniumhq.org/) has now emerged as one of the top contenders to take on QTP in the test automation tools space. Our teams at Zen Test Labs (www.zentestlabs.com) were one of the early adopters of this automation tool and have built up a decent level of expertise in automating test scripts using selenium. We recently also presented a tutorial on this topic at STARWEST, California in October 2011.

While both tools have their advantages and disadvantages, object repositories are one area that I have found to be important but not supported by Selenium. An object repository is a centralized place for storing properties of objects available in the application under test (AUT) to be used in scripts. QTP comes with its own object repository where one can record and store objects.

Over the course of a recent project I have tried to implement object repository in and it worked really well. I am listing below, how I went about doing this.

1. Create an interface of a name you want to give to your object repository as follows,
Interface ObjectRepository
{
// Now here you can store properties of the object in variable
Static String ddCategory = “id=ctl00_MainContent_ddlCategory”;
Static String btnSave = “”id=ctl00_MainContent_btnSave””;
}

2. Implement the interface into your class and you are good to use objects in your functions
Class TestLibrary implements ObjectRepository
{
public String testFuntion()
{
//here you can your object to perform actions or validations
selenium.select(ddCategory, “Testing”);
if (selenium.isElementPresent(btnSave)
selenium.click(btnSave);
}

}
In the above example, I have implemented it in java and have also tried it in C#. Thus, using OOPS concepts I feel that this can be implemented in any object oriented language supported by Selenium.
What is important to note is that one can also connect to excel or any database and store values. Thus in the event of changes in the application or changes in properties of any object; a simple change in the excel file or DB will reflect across all instances of the object.

Some of the advantages of doing this include:

1. Easily maintain your tests or components when an object in your application changes
2. Manage all the objects for a suite of tests or components in one central location
3. Code becomes more readable & easy to write when user defined objects name are used instead of complex and long property name & value.

These have been some of the approaches that have worked for me and my learning’s. Would be great to get to know other people’s experiences in implementing object repositories within Selenium

Hemant Jadhav | Senior Consultant | ZenTest Labs