Transform Data with Nodes

Learning Objectives

After completing this unit, you’ll be able to:

• List the types of nodes.
• Describe the uses of each node.

Swap Code with Nodes

Transformations typically require complex coding, however, DPE uses nodes to transform data in different ways. Whether you want to join two datasets, calculate a field value, or append a dataset to another dataset, there are different nodes available for each of these functions. Let’s review each of these nodes in detail.

Data Sources

To transform data, DPE needs to know where to extract data from. You can define the source object using the Data Source node. When you select the source object, you can also select its fields as data sources. For example, if the Contract object is the source, you can select all the Contract fields. Additionally, you can also select objects related to the Contract object, such as Account and Owner objects, and the fields related to these objects. 

If necessary, you can add up to 50 data sources in a DPE definition. All related objects have a lookup relationship with the data source object. For related object fields, you must specify a unique alias. This alias is used to refer to the field in the subsequent nodes where this node is referenced. 

Joins

As the name suggests, a Join node connects information from two nodes. From the records of the two nodes, select the fields that you want as part of the output, and map fields with the same data type. 

There are four types of joins you can choose from. You can have up to 20 Join nodes in a DPE definition. 

Join Type	Description
Left Outer	All fields from the first source node and all the matching fields from the second source node are part of the output.
Right Outer	All fields from the second source node and all the matching fields from the first source node are part of the output.
Outer	All fields from both source nodes are part of the output.
Inner	Only matching fields from both source nodes are part of the output.

Filters

You will likely have a lot of unwanted data to sift through. Adding filter conditions help you refine the dataset. Define filter conditions by either specifying a value or using an input variable. A Filter node can have up to 50 filter conditions. You learn more about input variables a little later in this unit. 

Appends

This node is used to create a union of nodes, by combining data from two or more nodes. You can append nodes that match these criteria.

• The field count in each source node must be the same.
• The field names of each source node must be the same.
• The fields in each source node must be sequenced to ensure that the data types match.

Groups and Aggregates

Group records by field, or aggregate the data from one or more fields. You can only group a dataset by fields of the type Text, Date, and Date/Time. In a Group and Aggregate node, you can group your dataset by up to 10 fields.

Formulas

Create multiple formulas and store the results of each formula in a new field. Before you create a formula, specify the alias, data type, and format of the derived field. The derived field can be of the following types.

• Date
• Date/Time
• Numeric
  • Specify the length and the decimal places of the numeric field. The length can be up to 17 digits.
• Text

Hierarchies

Determine a field’s hierarchy path based on the parent field you select. The Hierarchy node only provides the hierarchy path, not the field value that’s stored at every hierarchy level.

Slices

When creating the DPE definition, you may realize that you no longer require certain fields for processing. In such a case, use the Slice node to drop the fields from the subsequent nodes. First, select the fields that you want to keep. The remaining fields are then automatically dropped from the dataset created by the Slice node, though the dropped fields aren't removed from the source node. 

Writeback Objects

After the data is transformed, it must be loaded into an object within your org. Use the Writeback Object node to create, update, or upsert records with the transformed data. Create up to 10 writeback objects per DPE definition.

Input Variables

An input variable stores data to be referenced at different points in the DPE definition. It’s used in Filter and Formula nodes. If a field value is likely to change during a DPE definition, create an input variable for it and either provide a value or leave it blank. When you next run the DPE definition, you’re able to dynamically define or change the input variable value. This reduces effort and eliminates the risk of manually changing every instance of the value in the DPE definition.

An input variable can be text, number, date, date and time, or filter. For a Filter type variable, you must enter a valid JSON syntax.

In the next unit, you learn how to create a DPE definition with the help of an example. 

Resources

• Salesforce Help: Data Processing Engine Nodes