Optimize Data graph Performance and Credit UsageIf you aren’t familiar with or not sure when to use the Data Graph feature, you can first read this post. This article focuses on implementation best practices. As a refresher, data graphs combine and transform normalized table data from data model objects (DMOs) into new, materialized views of your data. Because the data is precalculated, you can make fewer calls and queries respond in near real time. However, how you define a data graph matters! If done well, it’ll help you achieve your goals.

Well-defined data graphs help you power your unique application needs.

Data graphs allow you to include the specific data model objects (DMOs) that you/your application cares about and the specific attributes within that DMO. They can improve performance by pre-calculating and denormalizing the data, allowing for faster retrieval of data. When defining your data graph, consider performance. Are you including only data that is relevant for the use case? Are you including data that can inordinately delay the refresh process? Because data is precalculated, it has a billing impact. Notably, billing is primarily impacted based on how many rows are processed while keeping the data graph up-to-date. Currently, data graphs are updated daily. As you define data graphs, make sure that the requirements for the use case are covered, the performance is good, and billing is optimal.

To do this, it’s helpful to keep these considerations in mind while defining data graphs. This allows you to meet the previous goals that we have set for ourselves!

1. Know when to use a data graph. Your application is repeatedly accessing a piece of data. This data/query requires multiple joins (across several DMOs). You don’t need the latest data—it can be a few hours old for your use case. This is an optimal situation that warrants a data graph. Because the calculation is done once and used multiple times, it’s more efficient billing-wise and it’s faster to retrieve.
2. Include only the needed data: A data graph allows you to choose the subset of the DMOs/attributes that matter to you. So choose wisely! Include only those DMOs/attributes/CIs/Segment Memberships that are frequently accessed. If there is additional data that is needed but not as often, you might want to exclude those from the data graph. Rather, you might want to query those DMOs separately. Keep the data graph limited to those DMOs that are accessed frequently. Note: you can combine the data from a data graph with data queried directly from DMOs at your application layer. This makes sure that only frequently accessed data is precomputed.
3. Consider breaking a big data graph into smaller data graphs: Because the data in data graphs are precalculated for faster access, it’s important to make sure that the data graph is the correct size. A bigger (and more complicated) data graph is predictably more costly to keep updated. In some cases, it can help to break the data graph into smaller ones. As mentioned before, some parts of the data graph could be updated less often or even separately. These are ideal situations to break the data graph into smaller ones. While doing this, you might want to consider breaking these data graphs into smaller ones based on how often the underlying data is refreshed. For example, if you have one big data graph containing the individual engagement data and sales order data, you could split it into two data graphs: one containing the individual engagement and another containing the sales order. The logic behind such a proposed separation is that individual web engagement changes might happen more frequently compared to purchases/sales for that user. By splitting this data graph into two, we aren’t forcing the sale order data graph to update every time a new user interacts with a web site.
4. Make sure data modeling is done correctly: When you build a data graph, you can add related DMOs based on relationships defined in the data modeling stage. When Data Cloud pre-calculates the data graphs, it joins the different DMOs using the join keys set in the modeling stage. If these relationships aren’t configured correctly, joins can take a long time to run, affecting performance. For example, if you had defined a N:1 relationship for DMO 1 to DMO 2 based on a field f and if f isn’t a primary key on DMO 2, the join will take longer time to run. This is especially relevant if you are working with custom DMOs. Make sure that the relationships are defined on the primary key appropriately as per the relationship type.
5. Prevent unbounded explosion while adding related DMO: While adding a related DMO into the data graph definition, watch out for related DMOs which have a 1:N relationship from the parent DMO to the child DMO. This is important because if there are multiple related records for a given parent record, it’ll all be included (unless it is an engagement type DMO where it’s limited). In such cases, data graph generation could fail if the total size of the record exceeds the current size limit (200 KB currently as defined in this article). If you aren’t sure about the potential limit on cardinality of such 1:N relationships (if you think it might go to 100’s of child records), it might be better to remove such DMOs from the data graph.

While considering the previous guidelines, make sure to review some of the guardrails in place for use of data graphs here. I hope these guidelines help you use data graphs to unlock the full power of Data Cloud!