trailhead

Prepare for the Security Review

Learning Objectives

After completing this unit, you’ll be able to:

  • Explain the purpose of the security review.
  • Outline what happens during a security review.
  • Summarize the scope of the security review.
  • List the tools that can help you prepare for the security review.
  • Describe what scanners can and can’t do for you.

Understand What’s Ahead

By now you’ve got the OWASP Top 10 list bookmarked. Maybe your team has even taken the Develop Secure Web Apps trail. You’ve designed your app in a way that comprehensively protects its data, and you’re building it by following secure coding practices.

You know that a security review process awaits your product, and you know that you can’t launch your product on AppExchange without passing the review. But what is the security review, exactly?

Test Your Defenses in a Safe Environment

You wouldn’t claim that a feature in your app worked properly without testing it, would you? Of course not.

One of your app’s most important features is its protection of customer data. At Salesforce, we believe that this feature is so important that we help you test it.

In a security review, our Product Security team tests your product’s defenses against the attacks described on the OWASP list. Our testers put on their burglar masks and try to break into your app in an intensive session that lasts several hours. Their mission is to steal data that they don’t have permission to access.

If at First You Don’t Succeed...

If our team successfully extracts data from your app, they send you a report listing the vulnerabilities they exploited. You can use this information to reinforce your app, and then it must go through the security review again. We explain this process in detail in a later unit. While it’s not the most fun ever, it’s a whole lot better than having real customer data stolen.

Like any testing scheme, the security review isn’t comprehensive. It doesn’t cover all possible vulnerabilities, and passing the review is not an ironclad guarantee that your app is completely secure. But just as testing your product makes you more confident about its quality, the security review provides you with vital feedback for bolstering your product’s defenses. There’s no such thing as guaranteed product security, but apps that pass our security review come pretty close.

We Review the Whole Thing

Maybe your app is a native app and lives only on the Salesforce platform. Or maybe it’s a composite app, with architectural components or services that live elsewhere in the cloud or on your company’s servers. The externally hosted pieces must be as secure as the rest. Otherwise, attackers can use them as entry points, just as a burglar can use a ventilation duct. All of your app’s components and services, no matter where they live, are fair game for the security review.

Here are some typical app architectures and the locations of their various parts.

App Architecture What Lives Where
AppCloud Off Platform Native Mobile App
100% native Custom objects
Custom Lightning components
Custom Apex code


Composite solution with external services Custom objects
Apex code
Services, such as maps, hosted on another platform, such as AWS or Rackspace
Composite solution with mobile apps Custom objects
Apex code
Workflows

Native apps for iOS and Android
API-only solution Connected App Stand-alone application that reads and writes data hosted on AppCloud. For example, an accounting solution that interacts with Accounts and Opportunities
Lightning Bolt Solution Lightning components
Apex code


Einstein Analytics app Apps
Dashboards
Lenses
Datasets
Dataflows
Custom objects


The Product Security team attempts to attack all of these pieces—not just those on the Salesforce platform. It digs deep and goes for your data everywhere. Plan accordingly! For example, native mobile apps can provide a great user experience, but they require extra work to protect them, depending on how you put them together.

Bring Everything to the Table

When you submit your app for review, make sure that you provide a complete test setup and instructions for using it. This setup must include a Developer Edition org with your managed package installed.

If your solution includes a native mobile app, include the installation link for it. If you’re integrating an external web-based accounting service, set up an instance that hosts it, or provide login credentials to a test account on the site. Overall, when deciding what needs to be provided, a good rule of thumb is to imagine that the Security Review Team is a potential customer who wants to test drive your whole solution end-to-end. When in doubt, refer to the security information in the ISVforce Guide.

Use Scanners to Conduct Your Own Review

If this security review process sounds a little scary, don’t worry. We want to see your product on AppExchange as much as you do, and we’re here to help. Aside from the resources we’ve mentioned, we provide scanners your team can use on your product to prepare for the review.

These scanners scour your app for specific vulnerability patterns. Scanners come in handy even early in the development process. We support three of them:

Scanner Description Use If
Checkmarx Scans applications hosted on the Salesforce AppCloud. Your solution has Apex code, Visualforce elements, managed packages. In other words, always use Checkmarx.
Chimera Integrates the best of open-source scanning technology in one service. Named after a mythical beast composed of the strongest parts of several animals. Powered by Heroku. Your solution has parts that reside on another platform that you control.
ZAP A free online scanner created by OWASP. Requires installation on a local system. Part of your solution is deployed on a domain that you don’t control.

These scanners generate reports containing all vulnerabilities they find in your product. Of course, they can’t find every vulnerability out there.

False Positives

Sometimes a scanner identifies an issue that isn’t really a problem. For instance, the scanner finds a vulnerability pattern that you have protected against, and it fails to recognize your method of protection. We call these errors false positives.

Let’s say you added some code to protect against malicious SQL queries, and the scanner doesn’t recognize that your code addresses this vulnerability. It reports a SQL injection vulnerability. You can interpret that report as a false positive.

If you think an error is a false positive, prepare a document explaining why, and include it with your security review materials. Be specific about how you’re protecting against the vulnerability indicated. It saves everyone time if we don’t have to chase you down and have you explain it to us.

How to Interpret Scanner Reports

Since the three scanners have different authors and run on different platforms, they generate different reports. This table shows you how to respond to the contents of each report.

Scanner Fix Ignore
Checkmarx Any errors classified as Low, Medium, or High. Informational warnings.
Chimera All errors except false positives. Low-severity warnings that you determine are false positives; explain why in a document with your review materials.
ZAP All errors except those listed in the next column. Once errors are fixed, provide a screenshot of the scan with your review materials, to verify the correct external endpoint is being scanned. These issues, which you can mitigate, but you are not required to fix. Be sure to document any other false positives.

Try to Hack Your Own App

Scanners are great, but there’s no substitute for human ingenuity. After you’ve fixed all the security problems you know about, try to find some more! Stock up on black T-shirts and sugary soda, and convert your testers into hackers.

With your team’s security advocate, devise a plan for adversarial testing, or attacking your own product and trying to steal its data. The OWASP Testing Guide is a great resource for this.

Give your tester-hackers a running instance of the app and let them loose with a mission to gain unauthorized access to customer or system data. Offer prizes for successful attacks.

Adversarial testing can turn up some surprising results. You might find yourself thinking, “I could have sworn we fixed that!” or wondering how your teammates got so good at hacking. Just remember, being hacked by them is a lot better than the alternative.

In the next unit, we step through the submission process itself.

Resources

retargeting