Test Continuously

Learning Objectives

After completing this unit, you’ll be able to:

• Explain continuous testing.
• Explain why continuous testing is important in quality assurance.

What Is Continuous Testing?

It must mean ongoing testing, right? While that’s part of it, there is quite a bit to dig into with this concept. Provar lets us know the International Software Testing Qualifications Board’s (ISTQB) definition is a good place to start. 

An approach that involves a process of testing early, testing often, testing everywhere, and automating to obtain feedback on the business risk associated with a software release candidate as rapidly as possible.

Let’s unpack that.

Testing Early
Testing early refers to the idea of “shifting left” discussed in previous modules. Continuous testing embeds quality throughout the software development lifecycle (SDLC). Waiting to test until all your software has been developed is postponing problems until the last minute. Keep in mind, though, testing early refers to when testing should begin, not when all of it should occur. Once you start testing, keep going! 

Testing Often
This is where continuous integration shines with the ability to get to faster feedback in smaller chunks. By integrating smaller batches of code into the source code repository, you can test your code more often, which helps you find and resolve any issues sooner. 

Testing Everywhere
Continuous testing means testing more than code. This even includes testing before you've written any code. For example, start by testing your requirements to ensure they cover business needs and are within the scope of the project.

Automating
Test automation is a key ingredient in continuous testing. Each time the new code is integrated with the source code, automated tests give fast feedback. Note that while automation is a big part of continuous testing, exploratory testing can be as well. Automation alone won’t cover “testing everywhere.” For example, testing ideas is a form of exploratory testing, and these are as helpful to test as your code. With automation, teams have more time to conduct this type of testing, which leads to better quality all around. 

Why Test Continuously?

Not only does continuous testing integrate quality across the SDLC, but it also provides a more complete picture of your code. Consider the different information you receive from a photograph versus a film. Sure, a picture might be worth a thousand words, but it can still leave a lot to interpretation. 

Now think of that image as a single test. You might get a good picture of what’s happening with your code, but you have to guess what was going on before or after you ran the test. Tests that pass at one single point in time only offer a blip of information compared to those that are run and pass time and time again. The more you are able to test your code, the more it can become like a film–a series of images, or information, that tell a complete story. 

That’s just an example with one type of test. Consider different tests that serve different purposes.

• Should the tests guide development before or during coding, or should they evaluate the code after it’s finished?
• Should the tests aim to prevent defects in the early stages, or should they find them after?
• Should the tests look at the external quality for users and business stakeholders, or should they check the internal coding and infrastructure?
• Should the tests focus on the intended value or just basic functions?

To answer the questions and be comprehensive, testing must be continuous or, as Provar points out above, early, often, everywhere, and automated. 

Sum It Up

Testing uncovers information. It clarifies ambiguities, defines implications, checks assumptions, and maps a path through the unknown. The more testing is integrated throughout the SDLC, the more it can reveal. 

Implement continuous integration with test automation, and you and your team will be in reach of continuous testing. While it is called the software development lifecycle, Janet Gregory perhaps better frames it: “Coding is really a small part of the whole delivery cycle while testing is throughout.”