Archive and Consolidate Salesforce Data
Learning Objectives
After completing this unit, you’ll be able to:
- Recognize which tools are useful to support consolidating orgs.
- Pinpoint when an archiving strategy is necessary.
Retaining, Archiving, and Accessing Data
Whether you’re using Sales Cloud, Service Cloud, Marketing Cloud Engagement, or you’ve built connected apps, a substantial amount of data is likely to be generated and pushed into your org at any given time. This presents some challenges. Data is gold, and deleting it can be out of the question because of its value. If you’ve ever encountered cases like these, implementing an archiving strategy may be beneficial.
- The amount of data in your org is causing poor performance, slow reporting, or record locking.
- Data retention policies aren’t being met.
- Global growth demands for reporting and new feature deployments aren’t being met.
- Data storage is a concern because of records that are stored but are not actively being used.
There are several ways data management on the Salesforce Platform can be controlled. A great way to archive your data is to use Heroku Connect. Let’s look at a use case to give you a better understanding of how to utilize Heroku Connect when you’re ready to archive data within your own solution architecture.
Use Case: Retaining, Tracking, and Managing Data
A large retail customer wants to archive old data to free up space in their org, and still be able to retain that data with the option to unarchive it for up to 7 years. To resolve these business challenges, Robert Bullard, from Get Cloudy Consulting, recommends using Heroku Connect, Postgres, and Salesforce Connect.
Within this architecture, there are three parts that are all processed within Heroku. The first is a web service. The web service provides endpoints for Salesforce to call for archiving and unarchiving records. These web services also expose operations that can be performed on the Heroku Connect Tables. The data is then added to a queue so it becomes an async process.
The second component is the Postgres database. A worker running in the background calls a set of stored procedures that process all the management of records between the archive tables and the Heroku Connect tables that are live.
Moving the data between the archive tables and the connect tables is as simple as using INSERT and DELETE. But if you’re interested in adding your own logic, you can use DML operations.
The third component is managed by the Heroku add-on, Heroku Connect. In this component there are two parts, the external objects and the sync engine. The external objects help to expose data in the archive table by an OData endpoint which can be consumed by Salesforce Connect and in turn exposes the data back to Salesforce as an external object. The second part, the Heroku Connect sync engine, does the bidirectional syncs between the live Heroku Connect tables within Postgres and the Salesforce org.
Solution Benefits
With this implementation, Robert expects significant performance improvements on his customers’ Salesforce org. This also removes the need for manually archiving data and assists the customer in faster modifications using the point-and-click integration through Heroku Connect.
Who knows, maybe sharing data between multiple external data sources could be the next endeavor for this customer.
Consolidating Multiple Salesforce Organizations
Most users have multiple orgs. If you’ve ever tried to share the data between them, you may have had some difficulties attempting to merge. One way to mitigate these problems is to use Salesforce Connect and the hub-and-spoke model. In this model, all of the data flows from the hub org to all of the spokes, or secondary orgs.
On the other hand, Heroku Connect also helps to sync multiple orgs data into one Postgres database. Once the data is in Postgres you have the choice to push the aggregated data set to a consolidated organization.
Use Case: Consolidating Siloed Department Org Data
A large retail chain has come to Cloudy Consulting in need of a way to centralize customer data from two sales orgs and a service org they own. Meanwhile, they are also interested in creating an individual customer experience for each line of business (LOB) they offer. After reviewing the customers needs, Robert decides to use Heroku Connect, Salesforce Connect, and the add-on Heroku Postgres.
In this architecture, there are three orgs—two sales orgs and a service org. Robert selects Heroku Connect because it gives customers the flexibility to centralize their data and the ability to create their own custom apps. Any processing or external code that needs to be written can be done directly in Heroku. Finally, Salesforce Connect pulls all of the CRM data from Heroku into the main org using OData API.
Remember Salesforce Connect uses OData 2.0, OData 4.0, Cross-Org, and Apex Custom adapters. Keep in mind when using either adapter, an org is limited to 20,000 OData callouts per hour and 1,000 OData callouts per hour for Developer Edition.
One More Thing...
Salesforce Connect can bring up to 100 tables of data from a single source integration. For example a customer is using multiple data sources for different objects.
Although there is no maximum on the amount of connections that you can have, there are Salesforce Connect licenses that are needed for each source you’re integrating with. If you have multiple source systems and you realize that the cost becomes overwhelming, the best way to leverage your resources is through Heroku using Salesforce Connect. Heroku Connect acts as an endpoint while interacting with multiple other systems. Salesforce Connect points to Heroku and pulls in the information you need. Although it requires more work to implement, it provides more scalability long term, and eliminates the need to create different connections for each platform.