Pressure to Release and the Impact on Testing

The pressure to release new products to a consumer with the appetite of a blue whale has led to an unprecedented situation. Companies are being pushed to embrace iterative and incremental development methodologies in a bid to keep users happy. Add what is now commonly known as the Google effect “I want it now and I want it free” and you have a disaster of epic proportions in the making.

I’d like to draw a parallel from the aircraft manufacturer industry.Airbus and Boeing have been in a battle for over 20 years now. This battle reached a crescendo in 2005 with the launch of the world’s largest passenger aircraft; i.e., the Airbus 380 releasing an extremely high pressure situation on Boeing to launch a new aircraft.Boeing worked on trying to get a better aeroplane that could beat the A380. Boeing finally delivered on its promise in 2007 with the launch of its own 787 Dreamliner. Unfortunately, Boeing has been plagued with a host of issues on the 787s including electrical fires on board mid-air, leading to the grounding of the entire fleet. Fortunately, there have been no fatal accidents to date.

Why do I state this example in a software testing blog?

The aircraft industry is a cutting edge world with great emphasis on passenger safety, deep rigor in testing all equipment thoroughly before allowing actual passengers to fly. I am very confident that Boeing would have followed a meticulous protocol to test this aircraft but somewhere the pressure to release has led to errors creeping in the process. Boeing’s issues reinforces the fact that we live in a world where the pressure to release can get to the best of us.Now this is an industry where testing is not taken lightly and this happened. Over the years, the kind of trade-offs I have seen software companies make I am surprised most of them have lasted this long 🙂

Given all these factors, we all acknowledge and accept that we cannot possibly test every point in a system to find defects. Thus one of the ways that has evolved of late is to run a “Risk based testing” program that protects you from critical failure in projects

Most customers I interact with end up using models that work on probabilities, impact, time and cost. One recommendation we end up making to most of them is to use statistical models to do this analysis. Do not rely only on past data, experience, user feelings, etc. Put it in a statistical model and see what the scenarios that come out are. Map these to your past data, experience, high failure areas, criticality to business, user feeling, etc. and add/ edit/ delete based on that. What you will end up with is a decent test plan that covers your risk sufficiently. It is critical that you revisit this periodically to adjust your plan on defects you are finding.

Ultimately, the decision to release a product is one driven by business needs. However, shipping a product that is not tested properly may not be that great an idea as a small number of people can make a lot of noise about problems-even those problems that are not so serious. Unfortunately that’s the world we live in now! As for Boeing, there is always the next battle to win in a duopoly .

Hari Raghunathan | AVP | Zen Test Labs

Advertisements

3 thoughts on “Pressure to Release and the Impact on Testing

  1. For the record, the Boeing 787 doesn’t compete with the Airbus 380. The 787’s design goal is fuel economy, not size. Boeing’s class competitor to the A380 is actually a variant of the 747. More likely the A380 was made in response to Boeing, and not the other way around.

    A bigger part of the reason the 787 has run into problems is not its lack of testing — it was delayed a number of times due to quality control, in fact (http://en.wikipedia.org/wiki/Boeing_787#Pre-flight_ground_testing) — but its use of new concepts such as composite materials, aerodynamics, and differences in engines and avionics and control systems, that complicated both production and testing.

  2. Thanks Keith for your reply and viewpoints. All your points are extremely valid (including the A380 vs. 747 competition). I stand corrected.

    My intention was to only emphasize on the pressure to release a product in an highly competitive environment. Additionally, as mentioned in my article, the airline manufacturing industry has one of the most robust QA and testing programs. The competition among Boeing and Airbus is legendary and was at its peak around these launches. In fact I think Boeing has a large order book for the 787 too and Airbus is trying to catch up with the A350.

    However, I would still like to believe that given overruns and delays Boeing was pushed to a release.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s