Choose the Right Big Data Solution
Unit 3: Choose the Right Big Data Solution
After completing this unit, you’ll be able to:
- Select a Salesforce data storage solution for a specific use case.
- Describe how to use Big Objects for big data.
Congratulations! You’ve just been hired by Get Cloudy, a high-tech consulting firm specializing in CRM implementations. Get Cloudy has built a successful business as a Salesforce partner, and it adds new clients every week. As a consultant, it is your job to recommend the best Salesforce data storage option to each of your clients. In this unit, you learn how to do just that.
Begin by asking a few questions: Should a copy of the data live in Salesforce? How much data are we talking about—over 20 million records? Does it need to be exposed in a customer-facing app? Once you know these things you can start to weed out solutions that don’t fit.
Evaluate Your Client’s Data Storage Needs
Where should the company’s data be stored? If it does not need to live in Salesforce and it already exists on a database with an OData adapter, Salesforce Connect is the best choice. If the data is part of a customer-facing mobile app, Heroku provides a platform to host it. You can also use Salesforce Connect to connect data on a Heroku Postgres database with Salesforce following a data proxy strategy.
If the data needs to live in Salesforce… how much? Anything over 20 million records calls for a Big Object. Otherwise, a standard or custom object is the easiest route.
How Data Is Used
How will the data be used? If it needs to be exposed in a customer-facing mobile app, Heroku provides a platform for app development. (The data is stored on a Heroku Postgres database and is synced back to Salesforce using a data replication strategy.) Otherwise, Big Objects is the solution.
Consider Data Analysis Options
Once your client determines what to do with their large volume of data, they have a few more things to figure out. Big Objects can store billions of records. How will the data be analyzed—in its entirety or in aggregations and subsets? Also, will you need to automate business logic for any part of the data?
Use Asynchronous SOQL and custom objects to create smaller datasets and aggregations. Use Tableau CRM if the entire dataset needs to be analyzed at once. If you want to take advantage of Salesforce automations over a smaller portion of the dataset, use both Tableau CRM along with Asynchronous SOQL and custom objects.
Now, Test Your Knowledge
As the newest Get Cloudy consultant you are ready to recommend data storage solutions based on each customer’s unique situation. Using the flowcharts as a general guide, what data storage solution would you recommend for each of the following fictitious companies? Don’t forget about configurability and maintenance (clicks vs. code), UI features, Salesforce licenses, and security requirements.
Frysco, a food distribution company, needs a data storage solution to track inventory, orders, and deliveries. Its customers are large grocery stores, and there are thousands of them. Frysco needs a mobile application for its delivery partners that addresses these requirements and constraints.
- A team of developers in the IT department is available for ongoing maintenance.
- All orders and deliveries must be retained for auditing purposes.
- Some rural grocery stores have no Wi-Fi and limited connectivity.
- Not every mobile app user has a Salesforce license.
Rockify, a digital music streaming service, needs a data storage solution for its customer data. They have 200 million individuals across the globe listening to songs and “liking” or “disliking” each song. The company needs to analyze data in meaningful ways, across different demographics and geographies.
- A Salesforce administrator and business analysts are available for ongoing maintenance.
- 25% of the 200 million individuals are weekly active users.
- All customer interactions must be retained for predictive AI tools.
Mango, an electronics manufacturing company, needs a data storage solution to track its inventory and assets. This needs to be made available to thousands of Mango’s partners around the globe.
- A team of developers and Salesforce administrators are available for ongoing maintenance.
- Partners need recent data about inventory and location (not in real time).
- Data is currently in an on-premise Oracle database, and reporting is very difficult.
- After implementation, incorporating as much business logic as possible into the app will be the next priority.
ETailored is a startup that has developed a digital personal shopper that’s accessible to its customers in an app. Its customer support reps use Salesforce to deal with customer issues and solicit customer feedback via surveys. Recently, the app has skyrocketed in popularity, and the data volume has increased with it.
- Standard and custom objects were used in the initial implementation.
- One administrator and one developer are available for ongoing maintenance.
- Survey grew from hundreds of thousands to millions per year.
- ETailored’s survey data must be kept for auditing purposes, but only data from the past 6 months needs to be available for immediate analysis.
||Heroku with data replication
||Designing a mobile app with Heroku makes sense when the users won’t have Salesforce licenses and may be without connectivity for long periods of time.
||Big Objects with Tableau CRM
||Analytics will need to be run on 50 million or more records at a time. The data should be stored in Big Objects and analyzed with Tableau CRM.
||Big Objects with Asynchronous SOQL and custom objects
||Once stored in Big Objects, smaller subsets of the data can be exposed in a partner community.
||Standard objects, custom objects, and Big Objects
||Stick with standard and custom objects, but add Big Objects to archive the data that is 6 months or older.
Now that you are familiar with a wide range of data storage options, you’re in a better position to choose the best solution for a specific set of needs.