Hello all,
I am looking for an advice regarding the consent in McNext.
I receive the customers consent via Zero Copy and I prepared the fields that McNext uses in Comm Subscription Consent object to determine if the communication will be sent and if its Marketing or Transactional, Engagement Channel (I mapped the channel IDs), Consent point value (email or phone) and adapted the consent status to OPT_IN or OPT_OUT. I think I figured this part correctly.
I prepared the object via the Batch Data Transform which can run hourly and it will consume credits each hour depending on amount of rows.
I want to try to use Streaming Data Transform but I cant use Zero copy data lake there. If I try to do it via Data Model object I cannot map the data from new DMO to Comm Subs Consent object.
The best approach I see here is to manage a consent object on source and handle the changes there or if is there some other Data360 feature that could help me achieve this. (a lot has been added and I'm exploring all the options that I have)
Is there anyone that had similar experience and if there is some advice that you can give on this topic?
Hi Marko - the reason you can't use a Zero Copy data lake object in a Streaming Data Transform is architectural, not a config gap, and it points straight at the fix.
Streaming Data Transform works on data as it's streamed/ingested INTO Data Cloud in real time (the real-time ingestion channels - Web/Mobile SDK, Server-to-Server / Ingestion API). Zero Copy data is federated: it stays in the external source (Snowflake/BigQuery/etc.) and is queried on demand - it never actually streams into Data Cloud as events. So there's no incoming stream for a Streaming Data Transform to act on, which is exactly why the zero-copy DLO isn't selectable as a streaming source and the mapping isn't available.
So for zero-copy consent data, Batch Data Transform is the correct/supported path - what you're already doing. To handle the cost and latency:
- Incremental batch: if you can key the transform to only changed rows (a watermark / incremental load) instead of reprocessing everything hourly, you cut credit consumption a lot - and tune the frequency to what consent actually needs.
- If you need true real-time consent (especially honoring opt-outs immediately): change the ingestion method for the consent signal from Zero Copy to streaming ingestion. Push consent changes into Data Cloud via the Ingestion API (streaming) or a streaming connector so they land as ingested/streamed data - which a Streaming Data Transform CAN then process into the Comm Subscription Consent object in near real time (streamed data processes asynchronously, roughly every few minutes). Keep zero-copy for the bulk/at-rest data and add a streaming feed just for the real-time consent updates - a hybrid.
That hybrid (zero-copy for federated data + streaming ingestion for the signals you must act on instantly) is the standard pattern when a use case needs both. One note: the real-time path is streaming-only - bulk isn't supported for real-time ingestion - so the consent feed would need to come through a streaming channel.
Refs:
- Real-Time Ingestion and Actions in Data Cloud:
https://www.salesforce.com/blog/real-time-ingestion/- Transform Data in Near Real Time (Streaming Data Transforms):
https://help.salesforce.com/s/articleView?id=release-notes.cdp_rn_2023_summer_streaming_data_transforms.htm&type=5
