Home » Continuous Delivery » The Absolute Need for Automated Testing

The Absolute Need for Automated Testing

Automated testing can radically increase the reliability of custom-developed applications and make it easier to add new features and components. The Protractor capability in Google’s AngularJS is a great way to put automated testing into practice.

Protractor is an automated testing engine for AngularJS, the Google developed framework for building enhanced HTML-based applications.  Protractor simulates user interactions and will help you verify the health of your Angular applications.  Using Protractor, you can radically expand the type and variety of automated interactive tests that you can perform, which makes your applications more reliable and reduces the amount of manual testing you have to perform.

We’ve been using Protractor for some our most demanding clients with great results.  My purpose today is to explain how you can use automated testing to protect against regression in your applications as you add more features and underlying complexity.  Future posts will delve into the specific uses and advantages of Protractor.

The Pain of Application Regression

Regression is the term for future code changes that break important features of an application.  Regression can cost your business time and money and create embarrassment in the marketplace.  The simplest way to guard against regression is to freeze development; however, this is rarely an option.  Users want new features and your competitors are only too happy to provide them.  You must either keep pace, or users will go elsewhere.

The Perils of Traditional Manual Testing

The traditional approach is to test all important new and existing features manually for every release.  This means that you must run every test against every browser that you support.  Next, you’ll need to run tests against popular operating systems and even against several different browser sizes.  Applications have more interfaces and back-end complexity than ever before.  Increasingly, manual testing processes struggle to handle the load, and it’s easy to skip key tests in the interests of time or because you believe a component of your software hasn’t be affected by your current development efforts.  The net effect is that manual testing misses key defects and can contribute to regression in your applications.

Embracing Automated Testing:  Shopping Cart Checkout

Automated testing is a better way to guard against regression.  Testing is work that is better left to a computer.  Using Protractor, you can build a much broader set of tests, execute them faster and more systematically, and generate more accurate results.  Let’s use a common scenario: Testing Shopping Cart Checkout Services.

The shopping cart checkout is a lifeblood feature for thousands of companies that do business on the web.  If it is broken, users can not buy from you.  Your revenue will stop.  Users will get frustrated, buy from your competitors, and might not return to your online shop.  You want to guard this feature against regression at all costs.  Regression can come in two forms:

  • Code Changes within your Control – As your developers add new features, automated testing can retest the application.  If automated testing detects a regression, you continue presenting the existing (unbroken) version of the application to the public while your developers rework the new feature.
  • Code Changes outside of your Control – Your checkout feature may work perfectly today but fail completely tomorrow.  This could be because the browser (e.g., Chrome, Firefox, Internet Explorer) changed some aspect that the checkout feature depended on.  You did not change any code, but the user perceives it as your application stopped working.  The checkout is broken and you are going to have to fix it.  The sooner you get started, the sooner your revenue will resume, and the fewer users will be lost to your competitor.  Periodic automated testing can give you a head start.

Key Considerations in Automated Testing

Executed on an Ongoing Basis
A regression can happen anytime since the last time your tests were run.  If you want to have confidence that there has not been a regression, you need to execute the tests regularly and often.  You must fully automate the test so that your computer can completely execute the tests by itself.

Protractor should run against a copy of your application in an environment isolated from your production server. Because protractor is executing features in the manner a real customer would, it is important not to create false data in the production system as a result of executing the tests.

Reproducible Tests
Your computer should be running these tests over and over again.  For the most part, exactly the same things should be happening.  The only time something different should happen is when your code changes – possibly indicating a regression.  Running the same test multiple times will not necessarily yield the same results.  When automating tests, you must keep these things in mind.

  • If your application enforces a uniqueness constraint (e.g., every user must have a unique id), then the second time you test the register user feature there may be an erroneous failure.
  • If your application depends on the current date (e.g., it validates that credit card expiry is in the future), then you need to account for this when you automate the test.

Simple Tests Are Better
The tests should be so simple that it is easy to look at a test and verify that it is correctly testing the feature it is supposed to be testing.  If it is complex, then you will need additional tests to verify your automated tests.  When you need tests to verify your tests, you are defeating the purpose of testing.

Cover Your Entire Application
If your suite of tests does not cover a feature, then it is vulnerable to regression.  If it is an important feature, then it should be covered by the automated tests.  It should be easy to verify that your suite of tests cover the features that you care about.

Provide Clear Reporting
The ideal report would be:

  • When there has been no regression: a green ball.  You can quickly look at the report, be confident that everything is OK, and work on other things.
  • When there has been a regression: a red ball followed by a description of the problem(s).  You are now in a position to assess the business impact of the regression and plan your response.

Coming Up Next

In the next blog post in this series, I will explain how, for applications development on the Angular JS framework, Protractor can serve this important function.

Scroll to Top