Test New Features Before a Release

Learning Objectives

After completing this unit, you'll be able to:

Identify the best testing approach for your organization.
Describe common automated testing methodologies and tools.

Identify Your Testing Approach

Right, so all these new features will be hitting your Salesforce orgs. Some will be auto-enabled by Salesforce, others you’ve earmarked specially to deploy. You may be wondering, “Is this going to all go without a hitch? Can I retire to a remote tropical island and leave my users to it?” Or perhaps you’re thinking, “Which business processes do I need to test? Do I need to test all of them?!”

These are all good questions. The answers depend on the complexity of your Salesforce org. Let’s refer back to our three trusty principles.

Forward plan.
Define roles, responsibilities, and processes.
Raise your game with advanced tools.

Forward Plan

Make sure you factor in time for testing activities ahead of a Salesforce release deployment. Consider the resources you need to devote—both the people and sandbox testing environments. If you have other Salesforce projects envisioned, you may want to adjust their schedules to incorporate some of the new release features. Barry decides to push out a project to onboard a couple of teams at Abacus by a month so the new users can be trained on the latest version of the platform.

Define Roles, Responsibilities, and Processes

In a simple point-and-click implementation, a single admin may be able to handle testing with help from one or two end users. If you have code in your platform, you need developer expertise.

Let’s talk process. What exactly does testing involve?

It’s important to bear in mind that any modifications to the codebase may inadvertently introduce regressions (that is, changes that negatively impact existing functionality). This is why our Salesforce product teams conduct rigorous regression testing to avoid this. So when we deploy a new release, most things will “just work.”

This means you should focus on testing any new features that impact the end-user experience, such as changes to the user interface, and any custom code you have built yourself.

Don’t forget you can take advantage of Salesforce’s sandbox preview to conduct tests in a sandbox org upgraded with the new Salesforce release before it’s deployed in your production org. If you’re not familiar with the sandbox preview, you can learn more in the Get Early Access with the Sandbox Preview unit of the Salesforce Release Readiness Strategies module.

When Barry was the lone point-and-click admin at Abacus, he worked with Kathy and Evelyn to test key scenarios. Now that Abacus has extended use of Salesforce, Barry has the support of the IT team. Together they have been able to explore how they can solve new use cases with code. To ensure everything runs smoothly, the team needs to formalize their approach to testing.

What Types of Testing Are There?

Let’s take a more detailed look at the main types of testing, so you can get a feel for the specific testing activities you need to plan for.

Unit Tests

Unit tests are key if you have done any kind of custom coding in your org. They focus on the smallest bits (also known as units) of low-level code you’ve added to your org. The aim here is to catch bugs early to avoid costly fixes later on.

Integration Tests

Integration tests apply if you have code interacting with anything not native in your org. Examples include displaying live weather or stock price data from a third-party service on a page layout, or exchanging information with your company’s billing systems. If you have multiple Salesforce orgs, test interactions between them.

Functional Tests

Functional tests verify that features work as defined in your technical requirements. These can be carried out by developers or end users. In small projects, they can be merged with user acceptance testing.

User Acceptance Tests (UAT)

User acceptance tests (UAT) give end users the opportunity to accept or reject the final feature delivery. You can couple this with end-user training if user feedback is relatively easy to implement.

To prioritize your functional and UAT testing efforts, a good rule of thumb is to focus on business processes meeting one of the following criteria.

High value: Critical to your business, such as creating opportunities or support tickets
High usage: Used by most or all users, such as a quota-tracking report
Both, such as login

Performance Testing

Performance testing evaluates how your system responds when large volumes of records or operations are being handled. For example, for a web service callout from your org, you could require a maximum response time of 10 seconds for a transaction processing up to 10,000 records.

Now that we’re familiar with the main types of testing, which type of tests should you use?

If you don’t have any custom code in your org, you can focus on functional and UAT testing using a manual approach.

If you have small amounts of code in your org (such as a few Apex triggers, Visualforce pages, or Lightning web components), you need to do unit tests in addition to your functional and UAT testing. You can build and run tests manually in the developer console.

Raise Your Game with Advanced Tools

If you have more complex custom code, then it’s time to reach for automated testing tools. In fact, if you have more than one developer working on Salesforce, we recommend using automated tools, regardless of complexity. These tools require some time investment, but ultimately help ensure you deliver high-quality code.

Automation tools execute tests automatically each time your codebase changes to check for any bugs that may have been introduced.

This kind of test automation is called continuous integration (CI). When you change your source code and push it to a source control repository (such as Git), your CI tool executes the scripts you’ve written for your unit, integration, functional, and UI tests. Example CI tools include Travis CI, Circle CI, and Jenkins.

How to choose a tool? Here are a few considerations.

Cost: Free tools versus paid. Paid tools typically include fewer limits on the number and duration of test runs, plus access to customer support.
In-house expertise: Some tools allow more customization options, but require a higher level of technical knowledge to use.
Test types available: Not all tools allow you to run UI tests or customize tests.
Reporting needs/alerting capabilities
Compatibility: The tool's compatibility with your organization's application lifecycle management systems, such as HP ALM or Jira.

Once you choose your CI tool, you need to set up a CI environment. Then you build custom jobs and tasks to automate your test scripts.

Tips and Gotchas

OK, you’re now ready to dive into testing. Here are a few things to be aware of.

Set Up and Manage Sandbox Environments

You need to create one or more sandboxes for testing. You also need to refresh and reset configuration to ensure reliable re-execution of your tests. For example, let’s say you have a unit test for an Apex trigger that searches for a particular record, and then creates a new one if it doesn’t exist. You need to delete the new record before running the test again to make sure it’s evaluating the functionality correctly.

This can quickly become time-consuming, so we recommend you use the command-line interface (CLI) or tooling API to create and manage sandboxes. Check out the How to Work with Developer Sandboxes and the Salesforce CLI blog post to learn more.

Test Data

If you’re working with developer sandboxes these won’t copy over any of your production data, so you need to create test data. Check out the Generate Data for Tests unit in the Unit Testing on the Lightning Platform to learn more.

Integration Testing

If you’re integrating with third-party systems, run tests on both sides of the integration.

API vs. UI Testing

Use API testing instead of UI testing wherever you can. UI tests can be costly to maintain since any UI changes introduced in a Salesforce release usually mean the tests need to be updated. You may be obliged to use UI tests in some cases, for example if you have a managed package and don’t have access to the source code.

If you’re working on a large project with many users, consider using a UI tool for dedicated UI tests that lets you evaluate different operating systems, browsers, devices, and accessibility requirements.

One such example is Selenium, an open source industry standard for UI testing. It also has the advantage of being non-Salesforce specific, so it can be used by developers who aren’t Salesforce specialists. You configure Selenium tests and the CI tool runs them in your environment.

Performance Testing

Make sure you don’t reach system governor limits during your performance tests. To avoid this, it’s important not to automate them. Make sure you notify Salesforce Support if you are conducting performance tests so they are not blocked by Salesforce.

Classic vs. Lightning

If you’ve developed tests for Salesforce Classic, bear in mind that some tests won’t run on Lightning. You may need to rebuild them.

A Few Final Tips

Test any Release Updates. These are listed under Release Updates in your org Setup menu.
If you spot a potential bug, check to see if it exists in Known Issues.
UI issues can sometimes be related to an end-user’s device, browser, or user profile, so test the most common combinations.
Test on a needs-driven basis. Budget in time to address any issues. A bug detected and fixed in the test phase costs significantly less than one resolved in production.
Be aware of false positives (that is, tests that fail even though the functionality works). You may need to test your tests!

Wrap Up

Whew, that was a lot to get through! Hopefully, though, you now have the key elements to map out a testing plan tailored to the needs of your org.

Besoin d'aide ?

En savoir plus