Learn Change Data Capture Characteristics

Learning Objectives

After completing this unit, you’ll be able to:

• Find out what objects are available for Change Data Capture.
• List the fields that are returned in the change event header and describe which fields are included in the body for each operation.
• Create a change event channel name.
• List the permissions needed for subscribing to change events.

Object Support

Change Data Capture can generate change events for all custom objects defined in your Salesforce org and a subset of standard objects. It supports change events for the most popular standard objects including Account, Contact, Lead, User, Order, OrderItem, Product2, and others. For a list of objects that support change events, see StandardObjectNameChangeEvent in the Object Reference for Salesforce and Lightning Platform.

To receive notifications for record changes, select the custom objects and supported standard objects that you are interested in from the Change Data Capture page in Setup. We go over the steps for selecting objects later in this module.

A Change Event Example

Remember Robert? His consulting client has defined a custom object called Employee__c that is part of a custom HR app that manages employee data. To synchronize employee data in Salesforce to the external HR system, Robert created an app that receives and integrates change events of new and changed employee records. 

Let’s look at an example of an event message that his app receives. It contains the data for a new employee record that a user created in Salesforce. Notice the changeType and entityName field values.

{
  "schema": "-pszPCNGMHqUPU1ftkjxEA",
  "payload": {
    "ChangeEventHeader": {
      "commitNumber": 65824495947,
      "commitUser": "005RM000001vI4mYAE",
      "sequenceNumber": 1,
      "entityName": "Employee__c",
      "changeType": "CREATE",
      "changedFields": [],
      "changeOrigin": "com/salesforce/api/soap/47.0;client=SfdcInternalAPI/",
      "transactionKey": "0005163c-8d04-d729-39bd-b917b035a66c",
      "commitTimestamp": 1569436136000,
      "recordIds": [
        "a00RM0000004ICOYA2"
      ]
    },
    "First_Name__c": "Jane",
    "Tenure__c": 2.0,
    "Name": "e-100",
    "Last_Name__c": "Smith",
    "CreatedDate": "2019-09-25T18:28:55.000Z",
    "LastModifiedDate": "2019-09-25T18:28:55.000Z",
    "OwnerId": "005RM000001vI4mYAE",
    "CreatedById": "005RM000001vI4mYAE",
    "LastModifiedById": "005RM000001vI4mYAE",
  },
  "event": {
    "replayId": 1
  }
}

The change event example message contains the employee fields first name, last name, and tenure, in addition to system fields like the creation date. For a new record, the event message excludes fields that a user didn’t set. For example, if the user doesn’t populate the tenure field, it would be excluded from the event message. 

The ChangeEventHeader field includes header fields that contain information about the event. A few header fields of note are listed below.

entityName

This field contains the name of the standard or custom object for this record change. In our example, it is Employee__c.

changeType

This field contains the operation that caused the change. For common change events, this field can have one of the following values. 

• CREATE
• UPDATE
• DELETE
• UNDELETE

In our example, the change event was for a record creation, so the value is CREATE.

changedFields

Use this field to find out which fields were changed in an update operation. This field is empty for other operations. Because the change event example is for a new record, this field is empty. If you update one or more fields, the field names, along with LastModifiedDate, are included in changedFields. For example, if First_Name__c is changed, the field value would be ["LastModifiedDate", "First_Name__c"].

changeOrigin

Use this field to find out what caused the change. If your integration app changes a record in response to a record change of the same object, you can get into an infinite cycle of changes. To avoid that, use this field to detect whether your app initiated the change, and, if so, do not process the change again and potentially avoid a deep cycle of changes. This field contains the Salesforce API and the API client ID that initiated the change, if set by the client. This field is populated for changes done by API apps or from Lightning Experience and is empty otherwise.

For example, if an app with clientID GetCloudy creates the Employee record via SOAP API, the changeOrigin field value is com/salesforce/api/soap/47.0;client=GetCloudy. In our example, the value is com/salesforce/api/soap/47.0;client=SfdcInternalAPI/, which means that the record change was done through the Salesforce user interface.

Transaction-Related Header Fields

The following header fields contain information about the transaction of the current change. 

• transactionKey
• sequenceNumber

Use the transaction fields to maintain an accurate replica of your org’s data in another system. The transactionKey field uniquely identifies the transaction that the change is part of. The sequenceNumber field identifies the sequence of the change within a transaction. The sequence number is useful for operations that include multiple steps, such as lead conversion or creating related records in an after insert  Apex trigger. If not all objects involved in a transaction are enabled for Change Data Capture, there will be a gap in the sequence numbers. We recommend that you replicate all the changes in one transaction as a single commit in your system. 

If you choose not to use a transaction-based replication process, your replicated data might not be complete if your subscription stops. For example, if your subscription stops in the middle of an event stream for one transaction, only part of the transaction’s changes are replicated in your system.

Event Replay ID Field

The last part of the event message contains a field called replayId. This field contains an ID for the event message that you can use to replay past events for up to 3 days. We won’t cover replaying events in this module. For more information, check out the Resources section.

Fields Included in the Body of an Event Message

The fields that Salesforce includes in an event message that a CometD client receives depend on the operation performed and the subscriber type. For example, for a new record, an event message received in a CometD client includes all non-empty record fields and system fields. However, in an Apex trigger or a Pub/Sub API client, the event message includes all record fields, whether empty or not, and system fields. For more information, see Change Event Body Fields in the Change Data Capture Developer Guide.

Order of Fields in the Event Message

The order of the fields in the JSON event message is not guaranteed. The order follows the underlying Apache Avro schema, a data serialization system that change events are based on. When an event is expanded into JSON format, the order of the fields might not match the schema depending on the client used to receive the event.

Merged Change Events

For efficiency, sometimes change events for one transaction are merged into one event if the same change occurred in multiple records of the same object type during one second. When change events are merged, Salesforce sends one change event for all affected records and the recordIds field contains the IDs for all records that have the same change. For more information, see Merged Change Events in the Change Data Capture Developer Guide.

Enriching Change Events with Extra Fields

Select fields to enrich change events delivered to CometD subscribers. The selected fields are included in the change event messages even when unchanged. For example, use enrichment when your app needs an external ID field for matching records in an external system. Or always include a field that provides important information about the changed record. For more information, see Enrich Change Events with Extra Fields in the Change Data Capture Developer Guide, and this Trailhead project: Create a Custom Channel and Enrich Change Events.

Other Types of Events: Gap Events and Overflow Events

Other types of change events are generated to handle special situations. Salesforce generates gap events when change events can't be generated or to inform subscribers about errors. Salesforce generates overflow events when the volume of changes is large. Gap and overflow events don't contain record data. Gap events include the record ID, enabling you to retrieve record data. For more information, see Gap Events, Overflow Events, and High-Level Replication Steps in the Change Data Capture Developer Guide.

Subscribing to an Event Channel

Now that you know what a change event message looks like, let’s find out how to receive the change events. Salesforce offers multiple ways to subscribe to a change event channel. For apps external to Salesforce, you can use Streaming API, or tools and libraries based on CometD, an open-source library that simulates push technology. Streaming API provides a subscription mechanism based on CometD. 

To process data changes asynchronously on the Lightning platform, write an Apex trigger for the change event. You learn more about Apex triggers for change events later in this module.

To receive instant notifications of Salesforce data changes in an app that is running on the Lightning Platform, you can use the empApi Lightning component.

You indicate which events your app is interested in getting by subscribing it to a channel. Your streaming app receives events in real time whenever a change occurs in Salesforce. 

If you would like to build your own app using Streaming API, check out the Streaming API Developer Guide. The guide includes code examples for subscribing using CometD. To build a Lightning Platform app using empApi, check out the lightning-emp-api Web Component or the lightning:empApi Aura Component documentation.   

Subscription Channels

A subscription channel is a stream of change events that correspond to one or more entities. You can subscribe to the channel to receive change event notifications for record create, update, delete, and undelete operations. Change Data Capture provides predefined standard channels and you can create your own custom channels. 

Standard Channels

The ChangeEvents standard channel contains change events from one or more selected entities in a single stream that you can subscribe to. If you expect change events from more than one entity, use the ChangeEvents standard channel. To receive change events on the ChangeEvents channel, select the entities for Change Data Capture. You learn how to select an entity for change events later in the next unit.

If you expect change events for only a single entity, use single-entity channels. With single-entity channels, you can subscribe to change events from only one custom object or standard object.

This table shows you how to specify the subscription channel corresponding to the records you’re interested in capturing changes for.

Custom Channels

Create a custom channel if you have multiple subscribers and each subscriber receives change events from a different set of entities. Also, use a custom channel with event enrichment to isolate sending enriched fields in change events on a specific channel. Custom channels group and isolate change events for each subscriber so subscribers receive only the types of events they need. Entities are automatically enabled for Change Data Capture when you create a custom channel that includes them. A custom channel has the following format.

/data/YourChannelName__chn

For example, if your channel name is SalesEvents, the subscription channel is:

/data/SalesEvents__chn

For more information, see Compose Streams of Change Data Capture Notifications with Custom Channels in the Change Data Capture Developer Guide.

Permissions for Receiving Change Events

Change Data Capture ignores sharing settings and sends change events for all records of a Salesforce object. To receive change events on a channel, the subscribed user must have one or more permissions depending on the entities associated with the change events. For more information, see Required Permissions for Change Events in the Change Data Capture Developer Guide.

Field-Level Security

Change Data Capture respects your org’s field-level security settings. Delivered events contain only the fields that a subscribed user is allowed to view. 

In the next unit, you learn how to use an open-source sample tool, EMP Connector, to subscribe to a channel and receive events.

Resources

• Change Data Capture Developer Guide
• Lightning Component Library: lightning-emp-api Web Component
• Lightning Component Library: lightning:empApi Component
  
• Streaming API Developer Guide
• External Site: CometD
• External Site: Apache Avro