Plan the Security Test
Learning Objectives
After completing this unit, you’ll be able to:
- Describe a security testing and evaluation (ST&E) strategy.
- Identify how ST&E specialists plan the strategy.
- Explain how ST&E specialists conduct predevelopmental testing.
Using a Security Testing and Evaluation Strategy
No matter how prepared your software development and IT operations (DevOps) teams are to develop secure systems, they won’t achieve their goal if an adversary is able to abuse the system to access sensitive customer data. This is why security testing and evaluation (ST&E) is so important to help reduce system vulnerabilities. Properly planning an ST&E strategy allows your organization to better understand the risks associated with a system’s complex structure, business capabilities, use cases, possible vulnerabilities, and security requirements.
One of the key parts of planning the strategy is to identify the acceptance criteria. The acceptance criteria states what is acceptable and what is not in terms of security. Acceptance criteria must be expressed clearly, in simple language, without ambiguity as to what the expected outcomes are. They must be testable: easily translated into one or more manual or automated test cases.
Your Roadmap to Security
The ST&E strategy provides evaluators and decision makers with knowledge to measure progress, identify problems, characterize system capabilities and limitations, and manage technical and programmatic risks. Planning a robust ST&E program also helps you measure technical progress and characterize operational security. The ST&E strategy provides a road map for evaluations, integrated test plans, and resource requirements necessary to accomplish the secure technology development objectives of the project.
You use the ST&E strategy to enable your team to produce secure systems that protect sensitive data. It serves as the basis for ST&E budgetary estimates. You also use the ST&E strategy to integrate developmental test (DT) and operational test (OT) objectives into a single test strategy. (You learn more about these tests in the following units.) This allows you to maximize efficiency during test execution while minimizing test resource requirements.
You leverage the ST&E strategy as a tool to involve testers and evaluators from within and outside the program early in the ST&E planning activities. You tap their expertise from similar experiences and begin pinpointing resource requirements. You also use the planning process to identify technological capabilities and limitations of alternative concepts and to design options under consideration to support cost-performance trade-offs.
Within the ST&E strategy, you describe the proposed concept for tests and evaluations throughout the program lifecycle, starting with technology development, and continuing through the engineering, manufacturing and development, production, and deployment phases.
Planning the ST&E Strategy
So where to begin? Let’s start by looking at an example. Amélie is an ST&E specialist at a government agency that provides supplies and services to other departments and agencies. She is part of a team that will test enhancements to a legacy system that allows departments and agencies to request supplies they need in one central location.
As the first step, Amélie works with Greg, the project sponsor, to plan the ST&E strategy. One of the most important features of the proposed system is that it should allow Amélie’s agency to enforce mobile device management (MDM) on provisioned phones. The team wants to implement security capabilities to ensure that the devices:
- Are covered by a mobile threat defense solution
- Require multifactor authentication (MFA) for transactions
- Prevent the execution of unauthorized software, among other capabilities
As Amélie begins to plan the ST&E, she recommends the best approach to evaluate the system’s ability to deliver these capabilities securely. She then works as part of a broader test team to formulate a testing plan and procedure. She knows that a key measure of the system’s success is its ability to securely provide usable functionality, so she involves users at various government agencies early in test planning to ensure the statement of desired capabilities is interpreted correctly and tested realistically.
Amélie also knows a key part of her job is to test whether the system can be exploited by an attacker to be used in unintended ways. She plans for ways to test for vulnerabilities; for example, she tests whether he can use an easily guessed authentication code, such as 0000 or 1234 to access the system.
Conducting Predevelopmental Testing
It’s time to get into the details of planning the testing by inspecting the deployment site, thinking about what changes will be necessary to integrate the new system functionality, and recommending software for testing. Let’s review some ways that Amélie conducts predevelopmental testing to set up the ST&E strategy for success.
The existing system the agency uses for MDM allows it to track the number of mobile devices provisioned to other agencies, but doesn’t provide the other security functionality the agency is hoping to deploy. Amélie works with his team to review and evaluate the existing system capabilities, and participates in preliminary design reviews with the development team to better understand the new system enhancements. Being involved at this stage means Amélie is able to help minimize the requirement gaps prior to development. She completes various activities, including the following list.
- Determine the scope, infrastructure, resources, and data sample size necessary to ensure system security requirements are adequately demonstrated.
- Decide on an appropriate level of test rigor for the system security enhancements.
- Conduct pretransition site inspections for the environment in which the system enhancements will be deployed, and develop site inspection reports to minimize installation and deployment issues and delays.
- Analyze and evaluate functional and operational changes that will be necessary to integrate the new system functionality with the legacy system in order to produce a single cohesive system.
- Recommend software for testing and security bug tracking, such as Xray, IBM Rational Test, and TestStand, to name a few.
Great job, security tester! You’ve planned a robust ST&E strategy. Next you prepare to dive into the work of further developing the ST&E strategy with data sources, a test plan, and a test environment.
Resources