Compare Data Storage Options

Unit 2: Compare Data Storage Options

Learning Objectives

After completing this unit, you’ll be able to:

• Describe the Salesforce data storage solutions.
  
• Match data strategies with business needs.
  

Overview

There are plenty of ways to store your data, but how much do you have and what do you need to do with it? In this unit, we describe each of the options based on where the data lives, how much data you have, and how it grows. Standard objects, custom objects and Big Objects are available technologies in the Salesforce first party data center. Data resides outside of the Salesforce data center with Salesforce Connect,  External Objects, and Heroku Connect. Let's take a deep dive into the differences.

Standard and Custom Objects

Whether a company is using Salesforce as a traditional CRM tool or a platform as a service (PaaS), its data is stored in objects. Standard objects come with Salesforce: Campaign, Lead, Account, Contact, Opportunity, and Case are some of the most common record categories. You can also build custom objects to store data specific to your company or industry. 

Use standard or custom objects when you need to:

1. Automate business logic with tools like flows, validation rules, and Apex triggers.*
   
2. Store fewer than 20 million records per object. The actual number may vary based on your use case.**
   
3. Access the latest data in real time.
   

* Standard and custom objects support transactionality across multiple records.

** If using standard or custom objects for large data volumes, follow Salesforce's best practices as outlined in the Large Data Volume (LDV) white paper and Trailhead module.

Salesforce Connect

Use this tool to connect outside data to your Salesforce org. (Note that there are a few features, like Sharing Rules, that don’t work with Salesforce Connect.) 

With Salesforce Connect, you can build external objects with clicks (not code), as long as your system has an OData 2.0 or OData 4.0 adapter. There’s also Apex Connector Framework if you do write code. Salesforce Connect makes it possible to search, view, and modify data that lives someplace else—all through the Salesforce UI.

Use Salesforce Connect when you want to:

1. Avoid copying lots of data into your Salesforce org.
   
2. Access small amounts of data at a time.
   
3. Get the latest data in real time.
   

Heroku Connect

Heroku Connect provides a bridge between Salesforce and the Heroku Postgres database, a managed SQL database-as-a-service for developers. Without it, data is displayed in a custom UI built on the Heroku platform. 

If you want to keep data in both the Salesforce and the Heroku Postgres databases, use a data replication strategy. Data replication with Heroku Connect can be one-way, from Salesforce to Heroku Postgres, or two-way. Data replication from Heroku Postgres to Salesforce can be configured to execute with very low latency, but not in real time.

If you don’t need to store any data in Salesforce, you can proxy data from the Heroku Postgres database into Salesforce using Salesforce Connect external objects. Remember that certain features of Salesforce aren’t available with Salesforce Connect.

Use Heroku Connect when you need to:

1. Use a business-to-consumer app that can potentially change the data stored in Salesforce.
   
2. Sync data between Salesforce and a Heroku Postgres database.
   
3. Store fewer than 20 million records per object.
   
4. Automate business logic with tools like flows, validation rules, and Apex triggers.
   

We’ve talked about managing data stored outside of Salesforce. Now let’s talk about managing an enormous volume of data.

Big Objects

As the name suggests, Big Objects store and manage a massive amount of data on the Salesforce platform. We recommend it for anything over 20 million rows, into the billions. Big Objects utilize a non-relational database to store data. They support real-time SOQL queries using primary key access. Primary key access is also known as the index, which is defined by custom fields in the big object. It’s important to consider which fields are indexed, how they are indexed (ascending or descending), and in which order. This ensures that the query completes across billions of rows.

Use with Tableau CRM

The Big Objects integration with Tableau CRM is limited to about 70 million rows per dataset from Big Objects. Take advantage of the declarative Tableau CRM UI instead of writing code to perform aggregate functions. Big Objects can be the source object in a dataflow. If the number of rows is too high for Tableau CRM, use Batch Apex to filter and store a smaller subset of data in a secondary Big Object or Custom Object for full Salesforce functionality. 

Use Big Objects when you need to:

1. Handle data in the millions or billions of rows.
   
2. Batch-process large sets of data and aggregations of the full dataset.
   
3. Analyze big data with operational reports or Tableau CRM.
   

Compare Solutions

Now that we’ve explored each data storage option, let’s compare them from a few different angles.

How Easy to Configure?

We’re talking about point-and-click vs. code here. Match your skills set to your solution. Remember to factor in the resources you need to maintain it. Here’s how it breaks down.

Displayed in the Salesforce UI?

Data stored in standard objects, custom objects, and Salesforce Connect can be displayed in any part of the Salesforce UI: page layouts, list views, reports, Visualforce pages, and Lightning Components. Heroku Connect uses standard and custom objects or Salesforce Connect (depending on whether it’s data replication or data proxy). Either way, data is displayed in the UI.

Big Objects are not automatically visible in the UI. But you have some options. They can be exposed in a Lightning Component or Visualforce page using Apex, SOQL, and primary keys. Typically, a smaller subset of data and aggregations is queried using Asynchronous SOQL, and stored in a custom object. This allows the subset to be displayed anywhere in the UI. Another option is to connect the entire dataset to Tableau CRM, which offers great reporting tools.

Meets Security Requirements?

Finally, remember to factor in specific security needs before choosing a data storage solution.

Resources

• Big Objects: Bring Big Data to Force.com (YouTube)
• Tableau CRM Limits (Salesforce Help)
• Large Data Volumes Module