Prepare for the Security Review
After completing this unit, you’ll be able to:
- Explain the purpose of the security review.
- Outline what happens during a security review.
- Summarize the scope of the security review.
- List the tools that can help you prepare for the security review.
- Describe what scanners can and can’t do for you.
By now you’ve got the OWASP Top 10 list bookmarked. Maybe your team has even taken the Develop Secure Web Apps trail. You’ve designed your app in a way that comprehensively protects its data, and you’re building it by following secure coding practices.
You know that a security review process awaits your product, and you know that you can’t launch your product on AppExchange without passing the review. But what is the security review, exactly?
You wouldn’t claim that a feature in your app worked properly without testing it, would you? Of course not.
One of your app’s most important features is its protection of customer data. At Salesforce, we believe that this feature is so important that we help you test it.
In a security review, our Product Security team tests your product’s defenses against the attacks described on the OWASP list. Our testers put on their burglar masks and try to break into your app in an intensive session that lasts several hours. Their mission is to steal data that they don’t have permission to access.
If our team successfully extracts data from your app, they send you a report listing the vulnerabilities they exploited. You can use this information to reinforce your app, and then it must go through the security review again. We explain this process in detail in a later unit. While it’s not the most fun ever, it’s a whole lot better than having real customer data stolen.
Like any testing scheme, the security review isn’t comprehensive. It doesn’t cover all possible vulnerabilities, and passing the review is not an ironclad guarantee that your app is completely secure. But just as testing your product makes you more confident about its quality, the security review provides you with vital feedback for bolstering your product’s defenses. There’s no such thing as guaranteed product security, but apps that pass our security review come pretty close.
Maybe your app is a native app and lives only on the Salesforce platform. Or maybe it’s a composite app, with architectural components or services that live elsewhere in the cloud or on your company’s servers. The externally hosted pieces must be as secure as the rest. Otherwise, attackers can use them as entry points, just as a burglar can use a ventilation duct. All of your app’s components and services, no matter where they live, are fair game for the security review.
Here are some typical app architectures and the locations of their various parts.
| App Architecture | What Lives Where | ||
|---|---|---|---|
| AppCloud | Off Platform | Native Mobile App | |
| 100% native | Custom objects
Custom Lightning components Custom Apex code |
|
|
| Composite solution with external services | Custom objects
Apex code |
Services, such as maps, hosted on another platform, such as AWS or Rackspace |
|
| Composite solution with mobile apps | Custom objects
Apex code Workflows |
|
Native apps for iOS and Android |
| API-only solution | Connected App | Stand-alone application that reads and writes data hosted on AppCloud. For example, an accounting solution that interacts with Accounts and Opportunities |
|
| Lightning Bolt Solution | Lightning components
Apex code |
|
|
| Einstein Analytics app | Apps
Dashboards Lenses Datasets Dataflows Custom objects |
|
|
The Product Security team attempts to attack all of these pieces—not just those on the Salesforce platform. It digs deep and goes for your data everywhere. Plan accordingly! For example, native mobile apps can provide a great user experience, but they require extra work to protect them, depending on how you put them together.
When you submit your app for review, make sure that you provide a complete test setup and instructions for using it. This setup must include a Developer Edition org with your managed package installed.
If your solution includes a native mobile app, include the installation link for it. If you’re integrating an external web-based accounting service, set up an instance that hosts it, or provide login credentials to a test account on the site. Overall, when deciding what needs to be provided, a good rule of thumb is to imagine that the Security Review Team is a potential customer who wants to test drive your whole solution end-to-end. When in doubt, refer to the security information in the ISVforce Guide.
If this security review process sounds a little scary, don’t worry. We want to see your product on AppExchange as much as you do, and we’re here to help. Aside from the resources we’ve mentioned, we provide scanners your team can use on your product to prepare for the review.
These scanners scour your app for specific vulnerability patterns. Scanners come in handy even early in the development process. We support three of them:
| Scanner | Description | Use If |
|---|---|---|
| Checkmarx | Scans applications hosted on the Salesforce AppCloud. | Your solution has Apex code, Visualforce elements, managed packages. In other words, always use Checkmarx. |
| Chimera | Integrates the best of open-source scanning technology in one service. Named after a mythical beast composed of the strongest parts of several animals. Powered by Heroku. | Your solution has parts that reside on another platform that you control. |
| ZAP | A free online scanner created by OWASP. Requires installation on a local system. | Part of your solution is deployed on a domain that you don’t control. |
These scanners generate reports containing all vulnerabilities they find in your product. Of course, they can’t find every vulnerability out there.
Sometimes a scanner identifies an issue that isn’t really a problem. For instance, the scanner finds a vulnerability pattern that you have protected against, and it fails to recognize your method of protection. We call these errors false positives.
Let’s say you added some code to protect against malicious SQL queries, and the scanner doesn’t recognize that your code addresses this vulnerability. It reports a SQL injection vulnerability. You can interpret that report as a false positive.
If you think an error is a false positive, prepare a document explaining why, and include it with your security review materials. Be specific about how you’re protecting against the vulnerability indicated. It saves everyone time if we don’t have to chase you down and have you explain it to us.
Since the three scanners have different authors and run on different platforms, they generate different reports. This table shows you how to respond to the contents of each report.
| Scanner | Fix | Ignore |
|---|---|---|
| Checkmarx | Any errors classified as Low, Medium, or High. | Informational warnings. |
| Chimera | All errors except false positives. | Low-severity warnings that you determine are false positives; explain why in a document with your review materials. |
| ZAP | All errors except those listed in the next column. Once errors are fixed, provide a screenshot of the scan with your review materials, to verify the correct external endpoint is being scanned. | These issues, which you can mitigate, but you are not required to fix. Be sure to document any other false positives. |
Scanners are great, but there’s no substitute for human ingenuity. After you’ve fixed all the security problems you know about, try to find some more! Stock up on black T-shirts and sugary soda, and convert your testers into hackers.
With your team’s security advocate, devise a plan for adversarial testing, or attacking your own product and trying to steal its data. The OWASP Testing Guide is a great resource for this.
Give your tester-hackers a running instance of the app and let them loose with a mission to gain unauthorized access to customer or system data. Offer prizes for successful attacks.
Adversarial testing can turn up some surprising results. You might find yourself thinking, “I could have sworn we fixed that!” or wondering how your teammates got so good at hacking. Just remember, being hacked by them is a lot better than the alternative.
In the next unit, we step through the submission process itself.
- OWASP Top 10 Most Critical Web Application Security Risks
- Introducing the Lightning Locker for Lightning Components
- Checkmarx
- Chimera Scanner
- Salesforce Live: External Integration Security with Chimera
- OWASP Zed Attack Proxy (ZAP)
- Security Requirements Checklist
- Required Reports & Test Environments
- OWASP Testing Guide
- Partner Security Portal
- Security Developer Center