Get Started with Data Processing Engine

Learning Objectives

After completing this unit, you’ll be able to:

• Explain how Data Processing Engine works.
• Enable user permissions and data pipelines.

Before You Start

Before you take the steps in this module, make sure you complete Salesforce Loyalty Management Basics. The work you do here builds on the concepts and work you do there.

Meet the Trusted Sidekicks of Cloud Kicks

Cloud Kicks, a stylish and comfortable custom sneakers manufacturer, is running a successful loyalty program called Cloud Kicks Inner Circle. The loyalty manager, Mary Levy, is always on the lookout for insights to improve the program. Since both the program and the business are largely data-driven, Mary works with Linda Rosenberg, the admin at Cloud Kicks, to make the most of their data.

Data analysis is critical to business growth and innovation. But gathering data and deciphering it isn’t always straightforward. Linda does her due diligence to find a tool that extracts and transforms data. Enter Data Processing Engine (DPE), a tool that helps transform data that's available in your Salesforce org and write back the transformation results as new or updated records.

Linda is happy because this seems to be exactly what she’s looking for. She sets out to learn more about DPE.

It’s All About Data

Data to a business are like cells to a living being—essential for its survival and growth. Marketing departments use data to narrow down their customer segments, sales departments rely on data to understand their customers’ needs, and business executives study market trends to tailor their strategies. Data drives every aspect of a business.

But with the vast amounts of data available from different sources, it’s only useful when transformed into actionable business intelligence. That’s where DPE comes in. It extracts data from different internal sources, transforms the data, and finally loads the newly transformed data into its destination. Let’s understand this one step at a time.

Identify the Data to Bring In: Let’s say you have to integrate data from different objects. First, decide what data to extract from which objects. At this point, the data may not be relevant to your purpose. It must be transformed into something you can work with.

Decide the Kind of Data Transformation Required: Based on what you’re looking to get out of the data, define the sequence of steps to transform the data. You can join, filter, group, and aggregate various fields and objects in a DPE definition. When you’re happy with the definition, run it.

Run the DPE Definition: At this point, Salesforce Data Pipelines kicks in. It’s a high-performance data platform that cleans, transforms, and enriches large volumes of data at scale. It syncs the source object data and transforms it based on the definition.

What happens to the transformed data? It’s loaded or written back to a destination or target object as new or updated data. This write-back happens on a particular schedule or when the definition is part of another business process.

By automating the process of manual, code-driven data integration, DPE improves efficiency and productivity and eliminates error probability.

Linda is excited to use DPE for data transformation, so she sets it up in the org. Let’s follow along as she does it.

Get Your Org Ready

To create definitions and run them, Linda must set up a few things, including Data Pipelines. Data Pipelines lets Linda query and compute the data available in the Salesforce org.

To find Data Pipelines in Setup, Linda must assign the Data Pipelines Base User permission set to herself. Here’s how she does it.

1. Click , and select Setup.
2. Enter Users in the Quick Find box, and then select Users.
3. Click the name of a user.
4. Under Permission Set Assignments, click Edit Assignments.
5. Select the Data Pipelines Base User and Data Pipelines Add On User permission sets.
6. Click Add, and then click Save.

After the data from the definition's source objects sync, a recipe is created to process the records. To view and edit the generated recipe, users require the Data Pipelines Add On User permission set. However, if CRM Analytics is already enabled in the org, users with the CRM Analytics permission set will be able to view the generated recipe. To learn more about recipes, check out the Data Processing Engine Definition Modeling Trailhead module.

She can now enable Data Pipelines.

1. Click , and select Setup.
2. Enter Data Pipelines in the Quick Find box, and then select Get Started.
3. Enable Data Pipelines.

The Data Pipelines Base User permission set and Data Pipelines are enabled. Linda requires a few other permissions and access to create, activate, and run DPE definitions.

Other Required Permissions

DPE definitions source data from a number of objects and fields. This data is transformed in different ways using nodes. For instance, the Join node is used to join two datasets. Linda needs access to objects and fields so that data can be used in the different nodes. Here’s a list of the types of access that are required and the actions they enable.

In the next unit, we explore the different nodes available to Linda to create a DPE definition.

Resources

Salesforce Help: Data Processing Engine