Optimizing Test Design

Boris Beizer, a definitive guru in the world of software testing, famously said “More than the act of testing, the act of designing tests is one of the best bug preventers known.”  Proactive test design can help you build quality into the system instead of testing for defects towards the end of the software development lifecycle. It allows testers to identify and eliminate defects before they are coded.

Test case design is also a powerful tool for risk analysis and optimization since it decides which parts of the system are to be tested. More often than not, the number of test cases created is far greater than the time and resources available to execute them. In such cases, good test design can be used to select the right number of tests that provide optimum coverage. Test design is also a huge factor in the success of any test automation project, even more so than tool selection and scripting.

However, a single test case design technique cannot be adequate to fully test a product. It is advisable to use a combination of these techniques because the value of each technique depends on the context of the functionality being tested. At Zentest, we use a combination of traditional and advanced test case design techniques to produce an optimized set of test cases that provide maximum test coverage. We have outlined these techniques below (click image to enlarge):

Img1

Advanced techniques such as Classification Tree are also immensely helpful in optimizing test cases. Classification Trees identify test relevant aspects which are called classifications and their corresponding values which are referred to as classes. The different classes from all classifications are then combined into test cases. Test design becomes visual through Classification Trees which makes communication and understanding easy. This technique provides the bare minimum test cases in a way that every ‘classification’ is covered at least once.

In the example below, a maximum of 12096 (2 X 8 X 7 X 2 X 2 X 2 X 3 X 3 X 3) test cases are required to test every possible combination. With Classification Trees, a minimum of 8 test cases and an optimal of 128 are enough.

Img2

Designing test cases is as much an art as a science.  At the core of good test design is a combination of test design techniques backed by creative and critical thinking.  The objective of good test design is to create reasonably sized tests that are aggressive enough to find defects. Reducing test design to a dull, mechanical activity will result in shallow and obvious tests that are unable to break the system and find defects.   In our experience, more than a specific technique, a combination of test design techniques backed by creative and critical thinking can lead to optimal results.

Zen Test Labs | Go-Live Faster

Deterministic QA: Optimal De-risking of Treasury and Cash Management Implementations

Banks invest in treasury and cash management applications every 18-24 months (includes replacements for outgrown or old systems or specialized functionality that bolts onto a current system). This not only means more work for the banks but also for their corporate customers that need to keep pace. Banks spend millions of dollars on these systems and implementation is complex. An average cycle ranges from 6 to 12 months based on the complexity and banks struggle with spiraling and unpredictable implementation time, costs while trying to keep up with increasingly frequent releases primarily because of QA and testing.

We have put together a whitepaper will help you understand common challenges faced in implementations, learn optimal approaches that minimize risk, balance requirements, customizations, quality, cost and time and understand risk mitigation strategies.This whitepaper will help you:

  • Understand common challenges faced when implementing a new or changing your existing cash management or treasury management application
  • Discover approches to balance quality, cost and time for such implementations
  • Learn optimal approaches that minimize business risk of a new technology
  • Understand high risk areas and risk mitigation strategies
  • Ensure the implementation and QA become predictable
  • Understand lessons learned from implementations

Download the whitepaper today!

Automating Without Access To The Application

I had the opportunity to be a part of a Corporate Banking project recently. I was involved in leading the functional automation testing part of the project. I found it to be a very exciting and challenging form of testing since our automation engineers did not have access to the application under test.  I wanted to share my experience in this post.

How can one automate an application without accessing the application? Confused?

The problem was that the client did not want to provide application access to the offshore team because of security concerns. We suggested setting up an isolated environment offshore, but unfortunately, that could not happen.

The deadlines were looming closer and we still did not have access to the application, so here’s what we did.

  1. We arranged a WebEx session with our onsite team member and asked him to record the functional flow using HP Unified Functional Tester (HP UFT) with the “record active screen” option “ON”.
  2. The recorded scripts were then transferred offshore.
  3. We opened these scripts offshore and started identifying objects with the existing recorded repository and also captured objects using active screen.
  4. Thus, our Object Repository was ready.
  5. After this, we started creating functions using the object repository and created test flows.
  6. After completing an automated flow, we transferred it to our onsite team member and ran the flow (without accessing the application).
  7. We were surprised that our entire flow executed successfully in a single run without any errors.
  8. We understood exactly what we needed to do in order to complete the project on time.

Completing this task was very easy for us since we already had a corporate banking repository of 4000 automated test cases and our own framework (ZenFRAME). We created Object Repositories and functions, attached those to the framework and updated the test flow in the framework …that was it! We delivered and deployed the framework in the client’s environment, setup the application URL and were ready to run our complete test suite.

The execution of our first test suite for a specific module in the onsite environment was very smooth. All the test cases executed successfully, and the result log was created in the framework.

So, this was the issue I faced and the solution I came up with. Please feel free to share any other solutions you know of for the issues mentioned above. Also, please feel free to share other issues that you faced and if possible provide the solutions that you came up with. It would be great to know other people’s experiences. Happy Testing! :)

Hemant Jadhav | Zen Test Labs

How to Setup an Automated Reminder in Tick Spot

Misplaced my timesheet

Timesheet is a mandatory implementation in every organization and implementation of timesheets definitely helps in reducing the frequency of disputes between employees and supervisors/employers. It is also a very effective way to track cost, making business accounting more accurate and error free.

Incomplete timesheets slow down our business. Employees spend longer trying to recall their jobs and hours, managers spend time chasing down incomplete timesheets, and accounting gets delayed. I understand your problems, and that is why I came up with this technique for setting up the automatic reminder feature in Tick Spot to complete timesheets on time. Follow some simple steps to set it up on your PC/Laptops.In one of my projects at Zen Test Labs, I implemented the automated timesheet reminder in Tick Spot. This helped not only the individuals working on the project, but also helped business in making correct effort calculations.

Setting up an Automated Reminder in Tick Spot:  

Every evening (you can specify your EOD), your browser opens up with the Tick Spot login screen.

Step 1

  • Create “timesheet.bat”
  • Open Notepad and type the following:
    • echo off
    • start /max iexplore.exe
  • Save the file as “timesheet.bat”

Step 2

Copy the file “timesheet.bat” to your preferred location

Step 3

Open the Run window and type “Task Scheduler”

Task Scheduler

Step 4: Add Batch Action

Select the Action tab, Click on “New”

Add Batch Action

Step 5: Add Batch File

To add the batch file, click “Browse” and select “timesheet.bat” from the stored location

Add Batch File

Step 6: Setup Trigger

To add Trigger tab and follow 5, 6, 7 & 8 actions

Click “New” –> Add the “Daily” radio button –> Add time –>Check “Enable” Check Box –> Click “OK”

Setup Trigger

Your automated reminder has been setup! The great thing about this feature is that it does not require any manual intervention and causes no overhead on system resources. You can extend this technique to other reminders too.

Mukund Wangikar | Zen Test Labs

OpenSTC, a CSR Initiative

OpenSTC..! What a memorable learning experience it was..! OpenSTC (Open Software Testing Course) is a CSR initiative by Zentest to train students in Software Testing regardless of their IT / Non-IT background. The course which is conducted free of cost is open to everyone who wishes to learn software testing. This three month course consists of a lot more than just theoretical training.

At OpenSTC, emphasis is placed on practical knowledge, which a candidate would require, not only to secure a good job, but throughout his/her career in the field of software testing.

The key technical areas covered in the course are:

  • Software Testing Life Cycle
  • Software Development Life Cycle Models
  • Principles of Software Testing
  • Test Case Design Techniques with a case study for designing Test Cases
  • How to Write Precise Test Cases (with ample case studies and examples to practice)
  • Writing End-to-End Scenarios
  • Decision Tables and Classification Tree
  • Defect Life Cycle and Defect Reporting

Trainers encourage students to extensively practice test case writing for multiple test scenarios, after which the team is introduced to the concept of Test Case Optimization. Identifying and logging defects is what excites testers most during the initial phase of learning. Defect Reporting and Writing End to End Scenarios are taught in such a manner that beginners would easily be able to write precise scenarios when they start working as software testers

The course teaches testers to be tactful explorers and to exercise good judgement. Memory and creativity based exercises are taught to improve thinking and analytical skills. After completion of the course, participants are offered a chance to work on live projects wherein they get to play the roles of Team Lead, Tester etc. and get an opportunity to apply their knowledge. The course also includes:

  • Seminars by senior members of the testing fraternity
  • Interview sessions with tips and mock interviews conducted by industry experts with feedback to participants
  • Improvement of communication skills
  • Self motivation exercises wherein participants learn about the deeper science of the mind, laws of attraction and how our thoughts shape our future. Participants are encouraged to channelize their thoughts, guided with ways to handle stress and live life with a positive attitude.

A crowd testing event was also hosted by Zentest which provided participants a chance to work on a real application in the testing environment. This day long activity was initiated by the team testing the application at Zentest and it comprised of understanding the application, exploring the application, testing and logging defects.

After completion of the course students can volunteer to be a part of the OpenSTC team. Volunteering at OpenSTC is another memorable experience wherein you can manage course activities and guide participants during live projects. Taking an initiative, communicating with the people at OpenSTC, co-ordinating and solving their queries helps one grow as a person and evolve as a leader.

Sonali Edake | Zen Test Labs

Test Automation is Not the Only Answer

I have worked on several test automation projects over the past few years. I also conduct test automation trainings as a part of our company’s CSR initiative and actively participate in online discussions about testing.  Since my work is centered on test automation, a lot of people frequently come to me with questions, some of which I would like to address in this blog post.

Question: “I recently graduated with a Bachelor’s in Computer Science. I am interested in pursuing software testing as a career. Can you recommend an automation tool that I can take up?”

Question: I am interested in software testing and have 3 years of experience in the BPO industry. I also underwent a 3 month QTP and Selenium training. Will this help me get a job as a tester?”

Question: “I have been working as a tester for the last 6 months. I want some growth in my career, so I am planning to move towards test automation. Which tools do you recommend I should learn?”

Question: “I have been working in manual software testing for over 4 years now. I think this has been a great mistake as far as my career is concerned. Most of my colleagues and friends are in test automation and it drives me up the wall .I also want to shift to automated testing; can you guide me as to how I should start?”

These and many similar questions have been asked frequently. All of these questions have a common line of thought: Test Automation. It makes me wonder if automated testing really is more important than manual testing!

Most testers I spoke to wanted to learn test automation only for the following reasons:

  • Knowledge of test automation tools can help their testing career and get them better job opportunities.
  • Some of them wanted to learn test automation just because their colleague was learning it!
  • Adding an extra point in their resume to make it stronger.
  • Highlight the fact that they learned automated testing in their performance appraisal meeting! :)

I am not against these testers but want them to realize that automated testing is not the only choice they can make to advance their testing career. Manual testing also offers a lot of growth. Knowledge of automated testing is definitely beneficial, but manual testing is also a very lucrative career path to pursue.

What I’m trying to say is that each of these roles – Manual Testing and Automated Testing have their own very unique challenges. Someone well versed with one role might not necessarily be well-acquainted with another. Treat yourself as a tester; not a manual or automation tester. Think of yourself as a tester with a set of skills, specialties, abilities and domain expertise.

Assuming that automated testing can replace manual testing and using automation tools without understanding testing & the underlying application can be very dangerous. Manual testing is not simple. It’s an art and requires high intelligence, creativity, judgment and skill with domain knowledge.

Finally, remember that human brains cannot be replaced by automated robots. :)

Any comments and suggestions are welcome.

Hemant Jadhav | Zen Test Labs

OWASP Testing 101 (Part 2)

In my previous post, I wrote about Broken Authentication, Session Management and Cross Site Scripting.  Today, I will continue talking about some more checkpoints to be kept in mind while performing OWASP testing.

Insecure Direct Object References
This involves modifying the URL parameter values and using them directly to retrieve a database record belonging to other users. If an ID or parameter in the URL is modified and refreshed, the application should not fetch a new record belonging to another user.This is the script we followed to test the vulnerability:

  1. Log into an application
  2. Navigate to the page where the value of a parameter is used directly to retrieve a database record, e.g. an invoice page with URL http://foo.bar/somepage?invoice=12345
  3. Modify the URL with a different invoice no. belonging to another user http://foo.bar/somepage?invoice=7985 and hit enter

Security Misconfiguration
Security Misconfiguration occurs due to poor configuration of an application (server or application level) which makes it vulnerable to malicious attacks. The application might be vulnerable to changes in website settings, unauthorized access or any other unintended actions on the application that divulge informative data or user details.This is how we tested the application for security misconfiguration:

Verify 404 Error message:

  1. Launch an application
  2. Manipulate the URL by deleting the directory structure and directly entering the page name
  3. Verify that “Server Error in ‘/’ Application” message displayed.The application should not return extra information related to the page or directory listings.

Intentionally crash the application using any of the following options where applicable and verify HTTP 404 Error:

  1. Change the DB configuration by providing invalid credentials OR
  2. Type only the domain name in the URL and hit enter
  3. Verify that error message is displayed

Sensitive Data Exposure
Even if an application is password protected, sensitive data such as credit card details, TAX ids and financial details etc should be encrypted or hashed in the database and masked while displaying at the front end. TLS/SSL should be used for transactions involving this type of data.This is how we tested the application for sensitive data exposure:

  1. Log into the application
  2. Navigate to My Profile / Password Reset page / My Account page
  3. Check the password field, Credit Card Number, SSN Number
  4. Launch the application with HTTP in the URL
  5. Check if the application is redirected to HTTPS
  6. The web application should be SSL Enabled and the URL should redirect to HTTPS

Make the following checks for sensitive data:

  • It should be masked in the application
  • It should not be cached
  • Auto complete should be disabled for forms containing sensitive data
  • CC/Account Number, Expiry/CVV Number etc., shouldn’t be exposed as clear text. Only the last four digits should be visible E.g. – **********1234
  • Account information (Account No., Routing No.) should be masked and stored in the database.  Account details should be masked on the receipt screen. E.g. – **********1234

Missing Function Level and Access Control
This is to verify user level access control of an application. Non-admin users should not be able to access screens that can only be accessed by admin users.

  1. Create two users, one with an admin role and another with a non-admin role
  2. Login as admin and verify that the application provides  functional and access privilege to  the admin user
  3. Login as a  non-admin user and verify if the restricted module is accessible

This project helped me experience a different flavor of testing and made me aware of the fact that applications are very vulnerable to malicious attacks and fraudulent users. If applications are not tested for security, then important user data and information is in danger of being compromised. Earlier, I used to test applications believing that functionality was the most important aspect, but now I have realized that for a robust and secure application, both functional as well as OWASP (security) testing are important.

Vasim Khan | Zen Test Labs