Connect to Multiple Data Sources and Create a Profile

Learning Objectives

After completing this unit, you’ll be able to:
  • Explain the difference between streaming event data and context data.
  • Create a data connection.
  • Explain the role of the partition key in connecting context and event data.
  • Create a data profile for your orchestration.

Streaming Event Data Versus Context Data

OK. Are you ready to start putting your business strategy into action for your Radical Routers? After thinking about the big picture for your Salesforce IoT solution and doing some upfront planning, you can start with the first step. And that step is connecting to the data that you want to bring into Salesforce IoT Scale Edition. In Scale, you connect to streaming event data (events) and context data (context).

Streaming event data or events can include data from devices, apps, and other things embedded with electronics, sensors, and software. It’s data about events that are happening to or within the device, app, service, or other object. In our use case for Radical Routers, the irregular heartbeat is the streaming event data that is most important to our simple IoT solution.

Context data or context is generally static data about the customer, app, device, or object. Examples are model number, version, contact name, service history, and so on. In our use case for Radical Routers, context data includes the name and email of the customer who owns the router.

Set Up Input Streams and Data Connections

In Salesforce IoT Scale Edition, streaming event data and context data can come from HTTP Push or Azure Event Hub connections, which are typical event data connections. You set up these connections in Input Streams. An input stream can have any number of data connections.
Two input streams: one for Radical Router context data and one for Radical Router event data.

For our use case we use separate input streams for our context data (1) and event data (2). This helps keep things more organized.

Within each input stream we’ve created an individual data connection that has a unique endpoint to which we send the data. And within each data connection, we’ve set up the schema for that data. You can either set the schema from a file or you can manually add fields. Below you can see the schema for the Radical Routers Events Data Connection.

Schema for the event data. Contains the following fields: device_id, irregular_heartbeat, IP_address, and status.

If you want to experiment and don’t have immediate access to your data, you can use third-party tools such as Mockaroo and Postman to create and push mock data into Scale. This allows you to test solutions before you deploy them with your real data. Scale accepts JSON (preferred), CSV, and TSV data formats.

Event Pipes and Data Pipes

Once you’ve set up your input streams and data connections, you need to send that data into a Salesforce IoT Scale Edition profile. You use event pipes and data pipes to send the connected data to a profile. Event pipes send streaming event data to the profile, and data pipes send context data to the profile. If you want to transform your data to make it just right for your orchestration, you can do that in event pipes and data pipes. Transformation is a fancy word for changing your data in flight. With Scale, you can do things like filtering, appending, and merging data; changing data formats; and other types of nifty data transformations.

Here’s a look at our event pipe for our Radical Routers event data.

Radical Routers Event Pipe with input stream sending data to the router profile.

In our event pipe, we keep things simple and just send the data to our router profile. Similarly, for our data pipe, we simply send the context data to the router profile without transforming it.

Here’s a look at the data pipe for our Radical Routers context data.

Data pipe for Radical Routers Context Data Pipe with four output attributes under the schema.

Once you’ve set up the correct event and context data to work in Scale, think about how to bring together, or marry, the right events and right context for your use case.

Let’s read about those nuptials.

Marrying Events and Context

One of the really cool things about Salesforce IoT is that it lets you bring together event data and context data. This lets you take actions not just on events, but on events in combination with context data about the device or the customer.

In Salesforce IoT Scale Edition, you bring together, or marry, event and context data when you create a profile. The profile lets you combine real-time events with stored context (which can be, for example, from an account record in Salesforce) to create a holistic data set for orchestrations. The event data and context data is only linked as it runs through Scale, and the profile doesn’t change the data inside or outside of Scale.

In the profile, you identify a partition key, which associates each instance of event data with its corresponding context record. The partition key is usually a unique identifier for the device, service, or app sending the event instance and its corresponding context record.

In our Radical Router use case, the partition key is device_id. Each event instance and each context record contain a device_id field. When the partition key value is a match, it combines the event data with the context data to apply it to the same router.

For example, imagine you have four events in your event data log that reference various status changes for a router with a device_id of A48993. Through the partition key, these events are associated with the context for device_id A48993. This lets you trigger actions such as automatically creating a service case for Awhina’s router if its heartbeat signal is inconsistent.

Event data and context data share a field with the same name that is used as the partition key.

Now you understand how the partition key associates each instance of event data with a specific router. Next we’re ready to put together the important data for Radical Routers, so that each router instance contains event data and context data about the router.

The name of the field/attribute you use as the partition key must have the same name in the event data and context data. For Radical Routers, we use the device ID as the partition key. The field has the same name in both sources. You cannot, for example, marry events and context using device_id and device_id_number.

Let’s look at how this data is brought together in a Scale profile.

Radical Router profile with event data fields on the left and context data attributes on the right, tied together by a common partition key: device_id.

The event data that you use is shown on the left (1), and the contextual data is shown on the right (2). You can see how the data is joined together by the specified partition key (3).

Now that we’ve got the data packed and ready to go, let’s send it on its way to carry out your business logic.