Explore the Cloned DPE Definition

Learning Objectives

After completing this unit, you’ll be able to:

• Explain the nodes of the Aggregate by Member DPE definition.
• List the use of each node type in a DPE definition.

Getting Familiar with the Builder

Rishi opens the Aggregate by Member_AMER Rebate Program Aggregation DPE definition from Setup and notes the key sections of the data processing engine page, or builder. On the left is a list of nodes and input variables, the building blocks of a DPE definition. But most of the page is empty, serving as the canvas for tweaking and customizing the definition. To find out what each section of the builder does, see Data Processing Engine Builder.

Rishi is curious to learn more about nodes. Let’s follow along as Rishi and Cindy explore nodes.

An Ode to the Nodes

Each node type in the data processing engine lets you transform your data in a different way. We talked about how a DPE definition is like an assembly line of production. What if you want to manufacture the same make and model of a car but with a few changes? Maybe you want to change the car color from red to black, add a sunroof, or update the seats with reclining levers. You can make these minor adjustments to your existing assembly line without changing the sequence of steps. You don’t have to start from scratch.

It’s the same for a DPE definition. If you want to make a small change to it, clone the definition, tweak the relevant node, and you’re good to go! By cloning the Aggregate by Member definition, Cindy has ensured that all the nodes, objects, and fields are carried over so she doesn’t need to start from scratch. She can simply focus on making those small changes to the cloned definition. 

Before Rishi and Cindy make the changes, they want to find out what each node of the definition does.

Here’s an illustration of the nodes that we cover and how they relate to each other. Don’t worry, we explain what they do and how they work together. 

Data Sources

Data sources list all the objects from which data can be extracted for processing.

• Program Rebate Type: Helps identify active rebate types and their specific IDs.
• Program Rebate Type Product: Helps identify the eligible products associated with the rebate type, if any. Eligible products are based on the included or excluded products in the rebate type.
• Rebate Member Product Aggregate: Helps identify the updates required in an aggregate record.
• Rebate Program: Helps identify active rebate programs and their specific IDs.
• Rebate Program Member: Helps identify the active rebate members for a program and their IDs.
• Rebate Program Payout Period: Helps identify the active rebate payout periods for a program.
• Transaction Journal: Helps identify the transaction journals that must be picked for processing data.

Now that the definition knows where to get the data from, it will filter the data to find relevant records.

Filters

Filter nodes help identify the criteria for selecting a set of records from each data source that we discussed. 

The following filters use input variables to search for a rebate type, program, member, and payout period respectively.

• Filter Program Rebate Type
• Filter Rebate Programs
• Filter Rebate Program Member
• Filter Rebate Program Payout Period

The actual variables can be populated when you run the Rebate Orchestration flow.

Here are some other nodes that help you filter records.

• Filter Journals By Usage Type: Ensures that only transaction journal records with usage type defined as Rebates should be picked up for processing.
• Filter Journals For Valid Products: Ensures that only transaction journal records for the eligible products are picked up for processing. Eligible products are based on the included or excluded products in the rebate type.
• Filter Active Rebate Types: Ensures that the rebate type record is active and that the record has the same ID as that of the current DPE. The ID value is an input variable.
• Filter Active Rebate Programs: Ensures that the rebate program record is active.
• Filter Active Rebate Program Members: Ensures that the member status of a particular member is active.
• Filter Active Rebate Program Payout Periods: Ensures that a specific payout period is active.

A few filter nodes use other formula fields as source nodes and come into play a little later.

• Filter New or Updated Aggregate Records: Ensures that valid new or updated records are inserted into the Rebate Program Member Aggregate object. It uses Compute New Or Updated Aggregate records as its source node.
• Filter Transaction Journals With Valid Activity Dates: Ensures that valid records in transaction journals have valid activity dates. It uses the Compute Transaction Journals With Activity Dates as its source node.
• Filter Valid Transaction Journals For Rebate Types: Ensures that valid records based on the rebate type filters are inserted into the Rebate Program Member Aggregate object. It uses the Compute Valid Transaction Journals Per Rebate Type as its source node.

Joins

When you know what type of records to pick for processing, use the Join node to join the data in fields from different records to match similar information. Here’s a list of predefined Join nodes.

• Active Program Rebate Type for Rebate Programs: Provides a list of active program rebate types for an active rebate program. Its source nodes are two filter nodes, Filter Active Rebate Types and Filter Active Rebate Programs.
• Valid Payout Period for Active Rebate Types: Provides a list of valid payout periods for the selected rebate types. Its source nodes are Active Program Rebate Type for Rebate Programs and Filter Active Rebate Program Payout Periods.
• Program Members for Active Rebate Type and Payout Periods: Provides a list of valid members for the selected active rebate types and payout periods. Its source nodes are Valid Payout Period for Active Rebate Types and Filter Active Rebate Program Members.
• Add Rebate Type and Payout Period to Transaction Journals: Provides rebate type and payout period information to transaction journals. Its source nodes are Filter Journals By Usage Type, and Program Members for Active Rebate Type and Payout Periods.
• Valid Activity Dated Journals with Program Rebate Type Product: Provides a list of valid transaction journal records with the eligible products. Eligible products are based on the included or excluded products in the rebate type. Its source nodes are Filter Transaction Journals With Valid Activity Dates and Program Rebate Type Product.
• Look Up Existing Aggregate Data: Looks up any previously computed aggregate data for the newly aggregated record. Its source nodes are Total Transaction Amount and Total Quantity, and Rebate Member Product Aggregate.

Formulas

Rishi and Cindy figured out which rebate types, programs, members, payout periods, and journals qualify for processing. But how would they know if an active journal record corresponds to the correct payout period, for a specific member, and within a specific program? They can create the following formulas and store the results of each formula in a new field.

• Compute Transaction Journals With Activity Dates: Identifies transaction journal records that have an activity date greater than or equal to the start date, and less than or equal to the end date of the payout period.
• Compute Valid Transaction Journals Per Rebate Type: Identifies valid transaction journals by applying rebate type filters. These filters are already part of the node Filter Active Rebate Types.
• Compute New Or Updated Aggregate Records: Identifies new or updated records to be inserted into the Rebate Program Member Aggregate object. The Look Up Existing Aggregate Data join node is used as the source node.

Groups and Aggregates

Using the following node, you can group a dataset using multiple fields and then aggregate the field values.

• Total Transaction Amount and Total Quantity: Calculates the total transaction amount and total quantity on the node Filter Valid Transaction Journals For Rebate Types. The results are grouped by member, rebate type, and payout period.

Writeback Objects

Having applied filters, joins, formulas, groups and aggregates, you can get a list of all valid records, new and updated. These valid records must be inserted into a target object, Rebate Member Product Aggregate. 

The transformed data is written back as results to the target object using the writeback object node, Upsert Total Transaction Amount and Total Quantity.

Cindy can also save and run the data transformation scenarios for a DPE definition from the Recipe page in Data Manager.

Rishi and Cindy learned a lot about the nodes in the cloned DPE definition. In the next unit, we see how they customize it.

Resources

• Salesforce Help: Data Processing Engine Workflow
• Salesforce Help: Data Processing Engine Builder
• Salesforce Help: Data Processing Engine Nodes