Hi Trailblazer Community,
We recently launched Einstein Decisions in Marketing Cloud Personalization using a Server-Side Campaign
, and I’m looking for insight from others who have experience with the initial learning/training period.
We have approximately 10 active promotions mapped to the same content zone and are optimizing toward Click
. Within the first 24 hours after launch, we noticed that one promotion was receiving a significantly larger share of impressions than the others.
We initially questioned whether this indicated a configuration or implementation issue, but after reviewing our Einstein Decisions Report, the behavior appears consistent with the model’s explore/exploit approach. Our understanding is that Einstein:
- Uses a portion of traffic for exploration, where promotions are served randomly to gather unbiased performance data.
- Uses the remaining traffic for exploitation, where the model serves the promotion it currently predicts will perform best.
- Continues collecting interaction data and periodically retrains, so promotion distribution may be heavily skewed—particularly early on—and can change as additional data is collected.
Our report also shows that the current winning model has a strong preference for one of the promotions, which appears to explain the impression distribution we’re seeing.
One additional observation: the training report included historical data associated with 15 promotions, although only 10 are currently active. Five promotions had been deactivated prior to our current launch.
Questions for the Community
For those who have implemented Einstein Decisions:
- Is heavily skewed promotion distribution expected during the initial learning/training period, even when the active promotions have the same eligibility/configuration?
- Is there a typical amount of time, number of interactions, or number of training cycles you recommend allowing before evaluating whether the model has sufficiently stabilized?
- How frequently does Einstein Decisions automatically retrain/redeploy its model, and is there anywhere in MCP where that cadence can be viewed?
- As additional interaction data is collected, should we expect the model to continue exploring alternative promotions, even if one promotion is currently being heavily favored?
- When promotions are deactivated, is their historical interaction data intentionally retained and used in subsequent model training? If so, how long can that historical data continue to influence the model?
- Are there specific reports or metrics you recommend monitoring during the initial learning period to distinguish expected model behavior from a configuration/eligibility issue?
At this point, we’re planning to allow the model to continue gathering interaction data and monitor subsequent training/model results rather than making premature configuration changes.
Would appreciate any guidance or lessons learned from others who have gone through the initial Einstein Decisions learning period.
