Build Testing into Your Pipeline
Learning Objectives
After completing this unit, you’ll be able to:
- Describe different types of tests you can use in your development process.
- Get started building quality and feedback into development.
- Explain why you should monitor your orgs for risky changes.
Different Types of Quality and Tests
As the team at Perpetual Learning focuses on improving their processes, they begin to think more about building in quality at every stage.
Looking at her team’s development metrics, Joan notices that a lot of work gets sent back to developers from the QA and release process, and that end users still find a lot of bugs.
Chuck, the QA lead, explains that some problems are functional quality issues, where something doesn’t do what it’s supposed to. But Praveen, the architect, notes that a lot of issues he sees are with structural quality. Developers can build code and configuration that work, but the product can still have problems that aren’t visible to users. Structural quality issues include code that contains duplication, is too complex to maintain, or causes security or performance problems.
Chuck points out that the same components are often responsible for repeated production failures. The support team wants those components to be retested before every release, but QA doesn’t have enough time to manually retest. They try to prioritize testing the main items. But they’re always rushing to finish so they don’t delay the release.
Praveen suggests they put more energy into automated testing. Instead of just trying to meet the minimum code coverage requirements for Apex, they can build tests to detect failures earlier in key support processes.
Copado allows them to auto-select Apex tests based on the classes and triggers included in a user story, and run those to give developers fast feedback. The same tests can be run automatically during deployments.
If they build a more comprehensive set of tests, they can run them before sending work to the QA team. Then the QA team can focus only on tests that must be done manually.
Chuck agrees, but he points out that it takes more time to write automated tests than to run tests manually a few times. They need to write tests selectively.
Joan asks the team to prioritize writing tests for the most critical systems, and to make sure those tests run in parallel to save time. Praveen offers to help divide the tests into different suites, so they can choose tests depending on the work developers are doing.
Building Quality In
Joan’s team has just released a new case creation form for their website. The developer, Anitha, built the form to the exact specifications of the support team, but the team learns from users that the form doesn’t make sense. They want to understand what went wrong.
Joan thinks back to the value stream map she made for her team. Her team always works in response to ideas from the customer support team. The product manager turns the ideas into a refined plan before a developer starts work. The QA team then tests completed work, and eventually the release manager releases it to users. If there’s a problem upstream, it affects all downstream departments.
In this case, perhaps there was a problem with the original idea or plan. And so even though Anitha built the right thing, she wasted her effort because the plan was wrong. Similarly, when development quality is poor, the QA team discovers many bugs during testing, or end users find them even later. The later someone discovers a defect, the more it costs to fix.
So the only way to ensure high quality is to build quality into the source.
That means it’s not only the testing team who has to think about quality. Developers need to build high-quality features to lessen the impact on testers. And planners have to make sure the team builds the right things, vetting ideas carefully to make sure bad ideas don’t get into the system at all.
Joan realizes that her team needs to give better feedback and communicate more effectively so they can build in quality. People are reluctant to raise concerns, and this leads to problems later in the process.
Most bugs are introduced during the coding process and detected during the testing process, but the cost of bugs increases the later they are detected. Data from Capers Jones, Applied Software Measurement: Global Analysis of Productivity and Quality (New York: McGraw-Hill Education, 2008).
Building Feedback Loops into the Development Process
Development techniques like static analysis, unit testing, coding standards, and code reviews are ways of building feedback into the development process to help improve the quality of the work.
Static Analysis
Static analysis is an automated way of identifying faults in code performance, style, or security. For example, static analysis rules can identify long classes or methods, because these can be hard to maintain. You can run static analysis quickly, inexpensively, and as often as needed, and get feedback faster than with any other type of test.
You can perform static analysis as one stage in a continuous integration (CI) process, but it’s most effective in real time in the developer’s code editor. This form of immediate feedback is called linting—it’s similar to spell-checking or grammar-checking. There’s no faster way to help developers see and act on feedback.
Take a look at some linting feedback on a snippet of JavaScript code.
The team at Perpetual Learning catches most issues using linting in their code editor. But they also scan their work using the source code analyzer PMD, which is built into Copado, to make sure they don’t miss anything. Copado shows which files contained violations, and which rules were violated.
Unit Tests
Unit tests are the best-known type of automated test. Technically, a unit test is a small, fast test that evaluates a particular unit of code. Not all tests written in Apex are truly “unit tests,” since you can also use Apex to test larger pieces of functionality.
While they’re writing code, your developers use unit tests to get quick feedback about whether their functionality is working and whether they’ve broken other code.
Whereas static analysis helps enforce general good practices (coding standards), unit testing checks the specific behavior of the system to make sure it’s correct.
Coding Standards
Coding standards are rules that grow out of collective experience and wisdom. These standards may vary slightly by project or programming language. Standards typically cover naming conventions, indentation, commenting, white space, bulkification, and the like.
Following standards makes every piece of your project’s code look and feel familiar. Consistent code is easier and faster for other developers to understand.
If your team consistently follows standards—and other teams in your organization follow them too—code can be more easily extended, refactored, and debugged.
Code Reviews
People don’t write code in isolation. They need support to do good work and make wise decisions. Code reviews provide a perfect opportunity for developers to give and get the support they need.
They’re also one of the most powerful methods of ensuring consistent, high-quality code, and they help you train new developers and ensure that more than one person is familiar with every line of code in the system. You can have senior developers do code reviews, or have peers review each other’s work. Developers can use pull requests to initiate a code review and to track feedback.
Setting Up Org Monitoring
Joan comes into work and finds Phuong, the IT security officer, talking to Tony, the release manager. They both seem stressed.
Apparently the Modify All Data permission was accidentally added to one of the profiles last month, but no one noticed until now.
Phuong points out that this is a compliance violation. The team needs to file an incident report.
Fortunately, Tony takes daily metadata snapshots of each org and determines that the setting was changed last month by one of the admins. There were many related changes at the same time, and it looks like this was an honest mistake.
Having snapshots is an important start, but Tony can’t manually review snapshot changes every day. Instead, the team wants to get alerts when something changes unexpectedly.
The team uses Copado compliance hub to create automated compliance rules to monitor for insecure permissions and other problematic metadata changes. They create an alert to ensure people on the team are notified if there’s an exception.
Copado compliance hub can also block deployments if they violate any of the rules. Now the team can integrate these checks into the development process. Phuong is relieved to adopt such automated compliance checks, since they’re a faster and more reliable alternative to manual change approvals. Compliance shouldn’t be a process that burdens the organization, but instead one that enables its success.
Conclusion
In this module, you learned about cultural aspects of DevOps, and two main ways you can improve your development practice: modular architecture and testing. Remember, take a scientific approach to continuous improvement. There are many things your team can do, but what matters is identifying which improvements will have the biggest long-term benefit for your organization. Then approach improvements systematically, experimentally, and with a sense of fun and adventure!
