Get to Know Security Testing and Evaluation

Learning Objectives

After completing this unit, you’ll be able to:

Describe the goals of security testing and evaluation (ST&E).
Explain the importance of ST&E.

What Is Security Testing and Evaluation (ST&E)?

Picture this: It’s Tuesday morning, and you sit down at your desk to start work. Your boss calls you into a meeting, and tells you your team needs to evaluate the security of a new system that stores financial data for your customers. You need to come up with a plan to test the system to ensure all customer financial data is encrypted at rest when stored in the system. Can it be done? Of course! How can you be sure? Because you’re an expert security tester and know how to harness the power of the broader evaluation team to verify the system’s security. In this module, you learn more about how to evaluate system security to ensure your customers’ data is protected.

Security testing and evaluation (ST&E) is the examination and analysis of the safeguards required to protect an information system as they have been applied in an operational environment. Put simply, you perform testing and evaluation to determine the security posture of the system. It’s a type of testing that uncovers vulnerabilities of the system and determines that the data and resources of the system are protected from possible intruders.

ST&E helps reduce threats and risks in systems to minimize the chances of loss resulting from a cybersecurity breach. Security is just like any other system functionality—you need tests that verify correct implementation. Security testing includes various methods, such as static analysis, dynamic analysis, interactive application security testing (IAST), and software composition analysis (SCA). These methods compare system capabilities against security capability requirements and specifications through testing to verify that the required capabilities have been implemented correctly. An ST&E specialist aims to detect defects or bugs both within the interfaces with other systems, and also within the system as a whole, using a series of different tests whose purpose is to exercise the full computer-based system.

A personal trainer helps a computer system exercise.

The Importance of ST&E

In your journey to become an ST&E specialist, you first need to understand why ST&E is so important. ST&E helps you proactively identify security risks and fix issues. It helps you avoid security emergencies in live systems.

The main goal of ST&E is to identify the threats to the system and measure potential vulnerabilities so they can’t be exploited. In the past, security was often addressed late in the development cycle. Today, best practices like development, security and operations (DevSecOps) integrate security testing at every stage of development. ST&E specialists help developers identify bugs, flaws, and other vulnerabilities early in the development cycle, giving them time to fix them before release. Fixing vulnerabilities early is far more cost-effective than catching them late in the development cycle. From requirements to design, coding to testing, savvy developers use the secure development lifecycle (SDL) to build security into a product or application at every step in the process.

The Seven Stages of the SDL

One way to remember how the SDL works is to remember that it consists of multiple stages. These stages form a structured process that creates a cycle.

Stage	What You Can Do
Learn	Learn the fundamentals of security and secure design by taking courses like this one. Use resources like those listed in this module to train yourself in SDL best practices. In your own organization, find out who the security teams are and what tools are available to help you incorporate security into all stages of development.
Design	Design with security in mind right from the beginning of your project. Find the security experts in your organization and partner with them to implement threat modeling to identify risks, create security acceptance criteria for each project, and define ownership up front.
Build	Build security as you develop. Then, be sure to mitigate vulnerabilities found during threat modeling and develop continuous integration and delivery practices.
Verify	Verify the security expectations you agreed on during the design stage. You may be able to sign off on this yourself for low-risk or low-impact projects. For larger projects, reach out to your organization’s security team for testing help.
Release	Release when you’ve completed a threat model and your threat mitigation is verified. In a cyclical development model, release is not the end of development—it’s a milestone in the process.
Own	Own your product in production. Ownership includes responsibility for patching, vulnerability management, and security incident response.
Reflect	Reflect as a team. Doing a retrospective with all participating teams is a great opportunity to document and discuss lessons learned during the project’s creation and release. These provide learning opportunities for future development work.

Test the Security of Your Systems

To better understand the importance of ST&E, let’s follow along with Seaghán, an ST&E specialist at a media company. Seaghán plays a critical role in developing secure systems. As an ST&E specialist, it’s his responsibility to understand who the project’s decision makers are and determine how and when to plan ST&E events so they are efficient and effective.

Seaghán is testing a system that scans his organization’s network to automatically discover hardware assets. He uses testing as a mechanism to assure it’s possible to accurately identify all hardware assets connected to the system. He also tests required capabilities, including security capabilities, such as the ability of the system to correctly identify malicious hardware, and block it from connecting to the network.

From the design phase of the project, up through coding, testing, and release, Seaghán works with the project manager, Manda, to manage the risks of developing, producing, operating, and sustaining the system, and to provide her with the information she needs to make important decisions. He and Manda define the security expectations for the project during the design stage, and use testing to verify the security expectations as the system is built.

An ST&E specialist and a project manager discussing security expectations (symbolized by a drawing on a whiteboard of a file folder with a key on it)

For example, Seaghán may discover during testing that the new hardware asset management system won’t integrate well with legacy products currently in use at his organization for identifying and updating out-of-date software on the organization’s hardware assets. This could lead to a failure to properly patch all the assets on the network. Unpatched software puts the organization at risk of an attack. He documents these findings and helps Manda think through the pros and cons of either 1) developing new features to better integrate the system, or 2) going live with the system as is, knowing the current shortfalls, and being accountable to the associated risk.

Seaghán knows that to be effective, testing can’t occur only at the end of development but must be addressed continuously throughout the entire lifecycle. He works with Gayle, the business analyst, and Michel, a software developer, to review and evaluate system requirements throughout the SDL. For example, Gayle may inform Seaghán that one of the key features of the system is that it needs to be able to detect which hardware assets are enabled with a specific internet communications protocol that is crucial to bolstering the organization’s security posture. Seaghán knows that this feature will need to be tested and evaluated, and can work with the developer to make sure it’s incorporated and deployed properly. Working together early in the project on defining these requirements saves costs and results in a faster deployment.

Seaghán doesn’t just have a role to play during requirements development. He’s involved in the system development all the way through until after the integration of all components of the system. He works with a team of evaluation specialists to rigorously test the whole system to ensure that it meets the specified business, functional, and nonfunctional requirements, such as the following.

The business requirement that the system can be integrated with other legacy systems that support software updates
The functional requirement that the system is able to block malicious hardware when detected
The nonfunctional requirement that the system runs a new hardware asset scan within a certain amount of time

Seaghán is an ST&E superstar! He recognizes the need for thorough, logical, systematic, and early test planning. He ensures that all testing is followed up with well-documented and unbiased ST&E results, aligning with security standards such as the OWASP Application Security Verification Standard (ASVS). These results provide useful information to system developers like Michel, decision makers like Manda, and most importantly, the system users, whether they be business users like Gayle or even outside customers.

Sum It Up

Now you understand more about ST&E. In the next unit, you learn more about the duties and qualifications of an ST&E specialist, and discover the skills that help ST&E specialists succeed.

Looking for Help?