Hi Team
I have published segment in SFMC from data cloud whihc uses relative attributes of DMOs. Because of these relative attributes the data is coming in JSON format.
I am using SSJS script to parse this data however script worked for few records but failing for large data. I have to parse around 3 lakh records or may be more than that.
Is there any other solution to parse segment data so that I can use it further.
,
You'll need to plan a logic to evaluate the number of iterations based on the total number of data received / 2500. Then using for loop, run the batches to flatten the JSON data out using a script activity. Implement a logging mechanism as well to log all the batch runs - success, failures etc. to gather as many details as possible for each batch runs, including retry mechanisms if possible.
Also, understand if the data can be processed incrementally for ongoing runs or not. And what's the max incremental load you can expect. This will give you enough info to plan your SSJS accordingly. If the load is too much, plan to chunk the data into 50-50% (as an example) and then allow your SSJS code to run on the chunked volume.
Hope this helps. Mark the answer as best if this is helpful for you.
Cheers,
Prithi
