I am using the Salesforce content data kit to ingest ContentDocument, ContentDocumentLink, and ContentDocumentVersion data for a Data 360 Search Index.
Salesforce Engineering confirmed that an initial or full index build processes all ContentDocumentVersion records included through the configured DMO relationship, not just each document’s latest published version. This means historical versions that remain in the DMO can be processed again during a new index build or rebuild, increasing Unstructured Data Processing credit consumption.
I have not found a supported declarative way to limit the ContentDocumentVersion DMO or Search Index to records matching the document’s LatestPublishedVersionId. Salesforce Support is investigating whether a Data Space Filter could help, but with the way I understand Data Space Filter that doesn't seem like a viable option (crossing my fingers).
Has anyone implemented a supported solution that ensures only the latest published ContentVersion for each ContentDocument is available to a Search Index? For example, have you successfully used a Data Space Filter, Data Transform, custom DMO, or a different data-stream design without breaking the standard file relationships required for indexing AND only indexes the latest ContentVersion data?
Hey @Santosh Narayan,
To clarify, I’m not looking to routinely rebuild the index. My concern applies to the initial build, because historical ContentVersion records may have accumulated in the DMO before the index is created.
For example, a 100 MB file could be ingested on day one and replaced on day three. When I create the index on day five, I only want the latest published version processed. However, Salesforce Engineering confirmed that my initial build also processed non-latest versions available through the configured DMO relationship, consuming credits unnecessarily.
I am ultimately trying to figure out how to exclude obsolete file versions from processing and credit consumption during the first build, and any future full rebuild.
