Transform and Write Back Data
Add a Join Node
In the previous step, you added the data sources. Next, add nodes to transform the data based on your requirements. For Cloud Kicks, add an inner join node type to connect the Filter Order node and the Get Product nodes. Next, use the OrderId field to map the two nodes.
- From Setup, in the Quick Find Box, enter
Data Processing Engine, and then select Data Processing Engine.
- Click Get Order Aggregate.
- On the Get Order Aggregate definition builder canvas, Click New Node.
- On the New Node window, specify these details.
- For Node Type, select Join.
- For Name, enter
Join Order.
- API Name: This field auto-populates with
Join_Order.
- Save your changes.
- For Node Type, select Join.
- In the Configuration pane, specify these details.
- For Join Type, select Inner.
- Under the First Node section, for Source Node, select Filter Order.
- Click Select Fields.
- In the Select Fields pane, find and select AccountId, OrderId, and TotalAmount.
- In the Select Fields pane, save your changes.
- Under the Second Node section, for Source Node, select Get Product.
- Click Select Fields and select Id, OrderId, Product2Id, Quantity, and TotalPrice.
- For OrderId, change the Alias to
ProductOrderId.
- In the Select Fields pane, save your changes.
- Under the Mapped Fields section, for both First Node Field and Second Node Field, select OrderId.
- For Join Type, select Inner.
- In the Configuration pane, save your work.
- In the builder canvas, click Save beside Activate.
The inner join node fetches all the records for the selected fields that have matching OrderId fields in both Filter Order node and the Get Product node.
Add a Group and Aggregate Node
Next, add Group By Fields that list the results based on the selected fields. Use the Product2Id field to group the results so that the output lists the total quantity and total revenue for each Cloud Kicks product. Add aggregation details to do calculations on selected fields as per your requirement. For Cloud Kicks, use the Count function to calculate the total number of orders, and the Sum function to calculate the total quantity and price for each order.
- On the Get Order Aggregate definition builder canvas, Click New Node.
- On the New Node window, specify these details.
- For Node Type, select Group and Aggregate.
- For Name, enter
Aggregate Order.
- API Name: This field auto-populates with
Aggregate_Order.
- Save your changes.
- For Node Type, select Group and Aggregate.
- In the Configuration pane, specify these details.
- For Source Node, select Join Order.
- Click Group By Fields.
- Select Product2Id and save your changes.
- For Source Node, select Join Order.
- Under the Aggregate Source Data section, add these details.
- For Aggregate Function, select Count.
- For Field, select OrderId.
- For Alias, enter
OrderId.
- Click Add Field.
- For Aggregate Function, select Sum.
- For Field, select Quantity.
- For Alias, enter
Quantity.
- Click Add Field.
- For Aggregate Function, select Sum.
- For Field, select TotalPrice.
- For Alias, enter
TotalPrice.
- For Aggregate Function, select Count.
- In the Configuration pane, save your work.
- In the builder canvas, click Save beside Activate.
The Group and Aggregate node groups the records by the Product2Id field and calculates the total number of orders. The node also calculates the total quantity and price of each order.
Add a Join Node
Next, add an inner join node to connect the Get Product2 node and Aggregate Order node. Then, use the Id and Product2Id fields to map the two nodes.
- On the Get Order Aggregate definition builder canvas, Click New Node.
- On the New Node window, specify these details.
- For Node Type, select Join.
- For Name, enter
Join Product.
- API Name: This field auto-populates with
Join_Product.
- Save your changes.
- For Node Type, select Join.
- In the Configuration pane, specify these details.
- Under the First Node section, for Source Node, select Get Product2.
- Click Select Fields.
- Select Id and Name.
- In the Select Fields pane, save your changes
- Under the Second Node section, for Source Node, select Aggregate Order.
- Click Select Fields.
- Select Field to select all the available fields.
- In the Select Fields pane, save your changes
- Under the Mapped Fields section, select Id for the First Node Field and Product2Id for the Second Node Field.
- Under the First Node section, for Source Node, select Get Product2.
- In the Configuration pane, save your changes.
- In the builder canvas, click Save beside Activate.
The inner join node fetches all the records for the selected fields that have matching Id and Product2Id fields in both the nodes.
Add a Writeback Object Node
Next, add a writeback object node and provide the target object and fields where you want to write back the transformed data.
In your Developer Edition org, we’ve created a custom object called Order Aggregates that includes the Total Quantity, Total Revenue, Product Name, and Total Orders custom fields. You can assign these fields in the writeback objects node to write back your transformed data and view the results.
- On the Get Order Aggregate definition builder canvas, Click New Node.
- On the New Node window, specify these details.
- For Node Type, select Writeback Object.
- For Name, enter
Writeback Order Aggregate.
- API Name: This field auto-populates with
Writeback_Order_Aggregate.
- Save your changes.
- For Node Type, select Writeback Object.
- In the Configuration pane, specify these details.
- For Source Node, select Join Product.
- For Target Object, find and select Order Aggregate.
- For Source Node, select Join Product.
- Under the Field Mapping section, add these details.
- For Source Node Field, select Quantity.
- For Target Field, select Total_Quantity__c.
- Click + Add Row.
- For Source Node Field, select TotalPrice.
- For Target Field, select Total_Revenue__c.
- Click + Add Row.
- For Source Node Field, select Name.
- For Target Field, select Product_Name__c.
- For Source Node Field, select Quantity.
- In the configuration pane, save your work
- In the builder canvas, click Save beside Activate.
Your Data Processing Engine definition is now ready. Here’s how the Get Order Aggregate definition with all the nodes configured looks like.
In this step, you learned how to configure nodes to transform the data and write back results in the definition. In the next unit, you learn how to run your Data Processing Engine definition and monitor the run’s progress.