Use Streaming API

Learning Objectives

After completing this unit, you’ll be able to:
  • Describe the primary benefit that push technology offers over pull technology.
  • Describe the various event products supported by Streaming API.
  • Explain the use of the event bus and how to specify replay options.

Streaming Events

To conclude our survey of Salesforce’s data APIs, let’s look at an API that serves an entirely different use case. Streaming API lets you push real-time notifications to clients using the publish-subscribe model. Some types of notifications are based on data changes in Salesforce. These notifications are possible with change data capture events and PushTopic events. Other notifications contain custom data that you define. They include platform events and generic streaming.

How does pushing notifications differ from the pull paradigm that our other APIs use, in which the client app requests, or pulls, data from Salesforce? Let’s examine the problem from a ship captain’s point of view.

Imagine that you’re sailing the high seas, and you want to keep an eye out for oncoming hazards, other ships, and islands rich with treasure. You put a sailor in the crow’s nest to keep an active lookout. Now put on your developer hat again. Let’s say you’re writing an app using REST or SOAP API that periodically checks to see if any accounts have been updated. You can use a similar solution and keep an active lookout by constantly requesting account data and checking to see if it matches the old data.

Now imagine that you’re on your ship again, but this time you have access to a shiny, new radar display. You don’t need to keep a sailor in the crow’s nest, because whenever an object of interest approaches, the display beeps.

Streaming API is your radar. It lets you define events and push notifications to your client app when the events occur. You don’t have to keep an active lookout for data changes or custom notifications—you don’t have to constantly poll Salesforce and make unnecessary API requests.

Streaming API can be used like a radar to detect data changes, and send and receive notifications

Tracking data changes in Salesforce is especially useful when you have business data stored in a system external to Salesforce. You can use Streaming API to keep your external source in sync with your Salesforce data with change data capture events and PushTopic events. Also, Streaming API lets you process business logic in an external system in response to data changes in Salesforce. For example, you can use Streaming API to notify a fulfillment center whenever an opportunity is updated.

In addition to data changes, you can use Streaming API to broadcast custom notifications with platform events and generic streaming. For example, an app can generate platform event notifications for orders that an order fulfillment service processes. Or an app can listen to platform events that Salesforce publishes to monitor user activity in Salesforce. With generic events, an app can listen to events and display a message whenever a system maintenance window is about to start or when a new offer is available to your users.

Streaming API and Event Products

Streaming API is a subscription mechanism based on CometD, which enables real-time streaming of event messages. CometD enables the server to push data to the client when the data is available and while the client maintains a connection to the server. 

Initially, Streaming API provided two types of events: PushTopic and Generic Streaming events. 

  • PushTopic events—PushTopic events are tied to Salesforce data. They enable you to stream Salesforce record changes to clients based on criteria you define in a SOQL query in a PushTopic record. The event messages include only the fields that you specify in the SOQL query. Record operations tracked include record creation, update, delete, and undelete.
  • Generic Streaming events—Generic Streaming events enable you to publish event messages with an arbitrary string value that the client receives.

Later, we introduced a second generation of event products. 

  • Change Data Capture—Change Data Capture is the new version of PushTopic events. With Change Data Capture, you get changes of records of all supported changed fields. Record operations tracked include record creation, update, delete, and undelete. There is no need to specify a SOQL query. Each event message contains header fields with information about the change.
  • Platform Events—Platform Events is the new version of Generic Streaming. With custom platform events, you can publish and subscribe to custom notifications. You can define the schema of the event data by creating platform event objects and fields. Also, you can subscribe to standard platform events that are defined and published by Salesforce to monitor user- and security-related activity in Salesforce and other things.
First and second generation event products

All the event products support the same CometD-based subscription mechanism, represented by Streaming API. Also, they all make use of the event bus, which is a service that enables the storage and retrieval of event messages. 

The newer event products, Platform Events and Change Data Capture, offer flexibility, scalability, and enhanced security. 

  • They support more ways to subscribe in addition to CometD. You can subscribe to platform events and change data capture events with Apex triggers. In addition, platform events support subscription with flows and processes.
  • The second-generation event products support encryption at rest of event data.
  • The versioned schema of a platform event or change data capture event enables subscribers to deterministically parse events. Each schema version corresponds to a unique schema ID, which is included in the event notification message.
  • Change Data Capture offers more standard objects than PushTopic events. Also, change data capture events offer more features, such as header fields that contain information about the change.

Salesforce continues to enhance Platform Events and Change Data Capture by adding new features to them. The old event products, PushTopic and Generic events, are still offered and supported but aren't on the roadmap for upgrades. To compare the features provided by each event product, see Streaming Event Features in the Streaming API Developer Guide.

Retrieve Past Notifications Using the Event Bus

Starting with API version 37.0, events are published to the event bus. Subscribers retrieve events from a channel on the event bus, including past events that are stored temporarily. The event bus decouples event publishers from event subscribers.

Salesforce stores PushTopic events, generic events, and standard-volume events for 24 hours and high-volume events for 72 hours. High-volume events include platform events and change data capture events. Standard-volume events are no longer available and include only events defined before Spring ’19. 

Retrieving stored event messages from the event bus enables you to catch up on missed events when the client was disconnected. 

Starting with API version 37.0, each event message contains a field called ReplayId. The ReplayId field value, which is populated by the system when the event is delivered to subscribers, refers to the position of the event in the event stream. Replay ID values are not guaranteed to be contiguous for consecutive events. For example, the event following the event with ID 999 can have an ID of 1,025.

A subscriber can store a replay ID value and use it on resubscription to retrieve events that are within the retention window. For example, a subscriber can retrieve missed events after a connection failure. Subscribers must not compute new replay IDs based on a stored replay ID to refer to other events in the stream.

In addition, there are other replay options, which are listed in this table.

Table 1. Replay Options  
Replay Option Description Usage
Replay ID Subscriber receives all stored events after the event specified by its replayId value and new events. Catch up on missed events after a certain event message, for example, after a connection failure. To subscribe with a specific replay ID, save the replay ID of the event message after which you want to retrieve stored events. Then use this replay ID when you resubscribe.
-1 Subscriber receives new events that are broadcast after the client subscribes. We recommend that clients subscribe with the -1 option to receive new event messages. If clients need to get earlier event messages, they can use any other replay option.
-2 Subscriber receives all events, including past events that are within the retention window and new events. Catch up on missed events and retrieve all stored events, for example, after a connection failure. Use this option sparingly. Subscribing with the -2 option when a large number of event messages are stored can slow performance.

This diagram shows how event consumers can read a stream of events by using various replay options.

Streaming events with replay options
Keep learning for
Sign up for an account to continue.
What’s in it for you?
  • Get personalized recommendations for your career goals
  • Practice your skills with hands-on challenges and quizzes
  • Track and share your progress with employers
  • Connect to mentorship and career opportunities