Connect to Multiple Data Sources and Create a Profile
- Explain the difference between streaming event data and context data.
- Create a data connection.
- Explain the role of the partition key in connecting context and event data.
- Create a data profile for your orchestration.
Streaming event data or events can include data from devices, apps, and other things embedded with electronics, sensors, and software. It’s data about events that are happening to or within the device, app, service, or other object. In our use case for Radical Routers, the irregular heartbeat is the streaming event data that is most important to our simple IoT solution.
Context data or context is generally static data about the customer, app, device, or object. Examples are model number, version, contact name, service history, and so on. In our use case for Radical Routers, context data includes the name and email of the customer who owns the router.
For our use case we use separate input streams for our context data (1) and event data (2). This helps keep things more organized.
Within each input stream we’ve created an individual data connection that has a unique endpoint to which we send the data. And within each data connection, we’ve set up the schema for that data. You can either set the schema from a file or you can manually add fields. Below you can see the schema for the Radical Routers Events Data Connection.
If you want to experiment and don’t have immediate access to your data, you can use third-party tools such as Mockaroo and Postman to create and push mock data into Scale. This allows you to test solutions before you deploy them with your real data. Scale accepts JSON (preferred), CSV, and TSV data formats.
Here’s a look at our event pipe for our Radical Routers event data.
In our event pipe, we keep things simple and just send the data to our router profile. Similarly, for our data pipe, we simply send the context data to the router profile without transforming it.
Here’s a look at the data pipe for our Radical Routers context data.
Once you’ve set up the correct event and context data to work in Scale, think about how to bring together, or marry, the right events and right context for your use case.
Let’s read about those nuptials.
In Salesforce IoT Scale Edition, you bring together, or marry, event and context data when you create a profile. The profile lets you combine real-time events with stored context (which can be, for example, from an account record in Salesforce) to create a holistic data set for orchestrations. The event data and context data is only linked as it runs through Scale, and the profile doesn’t change the data inside or outside of Scale.
In the profile, you identify a partition key, which associates each instance of event data with its corresponding context record. The partition key is usually a unique identifier for the device, service, or app sending the event instance and its corresponding context record.
In our Radical Router use case, the partition key is device_id. Each event instance and each context record contain a device_id field. When the partition key value is a match, it combines the event data with the context data to apply it to the same router.
For example, imagine you have four events in your event data log that reference various status changes for a router with a device_id of A48993. Through the partition key, these events are associated with the context for device_id A48993. This lets you trigger actions such as automatically creating a service case for Awhina’s router if its heartbeat signal is inconsistent.
Now you understand how the partition key associates each instance of event data with a specific router. Next we’re ready to put together the important data for Radical Routers, so that each router instance contains event data and context data about the router.
The name of the field/attribute you use as the partition key must have the same name in the event data and context data. For Radical Routers, we use the device ID as the partition key. The field has the same name in both sources. You cannot, for example, marry events and context using device_id and device_id_number.
Let’s look at how this data is brought together in a Scale profile.
The event data that you use is shown on the left (1), and the contextual data is shown on the right (2). You can see how the data is joined together by the specified partition key (3).
Now that we’ve got the data packed and ready to go, let’s send it on its way to carry out your business logic.