Develop the Security Testing Strategy

Learning Objectives

After completing this unit, you’ll be able to:

• Describe a security testing and evaluation (ST&E) specialist’s role in developing an ST&E strategy.
  
• Identify how to develop an ST&E program.
  
• Explain how to identify sources of data, write the test plan, and create a testing environment.
  

Develop Test Plans

Establishing a security testing and evaluation (ST&E) strategy helps you test the system’s security specifications and requirements. It minimizes the chance that an attacker could abuse the system to compromise its data. It also helps you describe in as much detail as possible the risk reduction efforts across the range of testing that will ultimately produce a valid evaluation of operational security before deployment.  

The goal of ST&E is to tell you about the effectiveness of the system in securing sensitive information. Similar to a school environment where a test the students take informs the teachers about the effectiveness of their teaching, in ST&E, a test helps the tester and developers, and project sponsors make a judgement about the security of the system.

The ST&E strategy should include these items. 

• The objectives of the evaluation
  
• A description of the system, including the mission, concept of operations, major performance capabilities, and possible threats and vulnerabilities
  
• Expected mitigations and outcomes
  
• Capabilities, in part defined by what threats need to be protected against
  
• An identification of and plan for management of technology risk
  
• Schedule and funding estimates
  

This establishes an early consensus on the scope of how the system will be tested, with particular consideration given to needed resources in order to support testing activities.

A Day in the Life of an ST&E Specialist

ST&E specialist Lee is working to identify key evaluation points, questions, and outcomes for the security test of a new system that contains data about hazardous waste disposal. Because the system contains highly sensitive data, Lee needs to evaluate points such as the following. 

• Whether the system is properly segmented from other systems in the organization’s network
  
• What the impact would be if the confidentiality of the system was compromised
  
• Under what scenarios an attacker would be able to impact the integrity of the data
  
• Whether changes made to address identified vulnerabilities introduce any new vulnerabilities—a concept known as regression testing
  
• Whether the errors and bugs identified during initial testing are in fact fixed
  
• Whether the system is both usable and secure
  

Lee knows that in order to address these questions, the ST&E strategy should be integrated throughout the development process. He will use the strategy to provide essential information to decision makers, assess attainment of security performance parameters, and determine whether the systems are operationally secure. He’ll also confirm performance against documented capability needs and adversary capabilities as described in the system’s threat assessment. 

Design an Analysis Structure, Write the Plan, and Create a Testing Environment

Design a Data Analysis Structure

As a next step, Lee identifies likely sources of required data and designs a data analysis structure (that is, the types of data his test must generate and how to analyze that data). Examples of data analysis structures include the following.

Lee captures the data used to evaluate the system’s security and readiness for production and deployment. He uses ST&E to better understand how securely the system is performing during development, and if it is ready for fielding. He clearly documents all sources of data and the data analysis process for consistency, so that the development teams are able to identify and learn from his findings.

Write the Plan

Next it’s time to document the test plan. Lee knows the test plan should be applicable and complete, and document the overall ST&E program. By documenting test procedures, he ensures replicability and compliance with standards. He writes a formal description of how he plans to approach testing in terms of resources, infrastructure, the overall IT security strategy, key stakeholders, effort, time, and approval processes. He makes sure to define testing events, releases, and levels of testing.

Create a Test Environment

Next, it’s time to create a testing environment, generate a test case, and generate the data to be tested. Lee knows that testing activities need certain environmental factors, such as servers, frameworks, hardware, and software, for executing developed test cases. Software and hardware configuration along with test data setup are the main components of this phase.

Lee wants to ensure the system containing data about hazardous waste disposal is secure, and that it requires all users to authenticate through multi-factor authentication (MFA) using a hardware token and personal identification number (PIN) for access. To set up the testing environment, he makes sure the software and hardware for the testing team is available to execute test cases. He does this by:

• Setting up the system itself, including test data on a sample population of users.
  
• Properly configuring the database server that underlies the system’s authentication mechanisms.
  
• Ensuring the front-end environment where the users enter their credentials is set up.
  
• Equipping the testers with bug reporting tools, such as Bugzilla, BugHerd, or Mantis Bug Tracker, to name a few.
  

As the last step in developing the ST&E plan, Lee submits the plan to the project sponsor for approval prior to executing the plan. Throughout developing the plan, he has involved testers, evaluators, and others to provide the necessary technical, operational, and programmatic expertise to ensure nothing is overlooked in laying out a complete strategy.

Knowledge Check

Ready to review what you’ve learned? The knowledge check isn’t scored—it’s just an easy way to quiz yourself. To get started, drag the description in the left column below the matching term on the right. When you finish matching all the items, click Submit to check your work. To start over, click Reset.

Great work! You’ve now learned how to develop an ST&E strategy and build in security at every step of the process. Next, it’s time to verify whether the system capabilities that your team implemented are secure.

Resources

• Trailhead: Threat Intelligence
  
• External Site: MITRE: ATT&CK