You configure an A/B test node with a specific variant split, but when you check the node stats later, the reported ratio doesn't match your configuration. This article explains why this happens and how to identify the root cause.
Understand how variant assignment works
The platform assigns an A/B test variant to a customer profile only once. For a scenario A/B test node, this means the following:
- The platform assigns the variant the first time a customer profile arrives at the node.
- If the same customer profile passes through the node again, for example, through a repeated scenario trigger, the platform reuses the original variant instead of assigning a new one.
Understand why the stats differ
Node stats report customers who pass through the node, not those who receive a fresh variant assignment. Because repeat visits reuse the original variant, the ratio in the node stats can drift away from the ratio you configured.
Most of the time, the root cause sits in your scenario logic. If your scenario logic disproportionately sends customers from one variant back into the A/B test node more often than from other variants, the node stats reflect that imbalance rather than your configured split.
Example
The following example shows an A/B test node configured with a 90/10 split:
- test-channel=email: 90%
- control-channel=email: 10%
- Control Group: 0%
The node stats for this node show a completely different picture:
- test-channel=email: 12.5% of the last run, 28.7% of the total
- control-channel=email: 87.5% of the last run, 71.3% of the total
The scenario logic sends the control-channel=email variant back through the node far more often than the test-channel=email variant. Each pass-through reuses the customer's original variant, which pulls the reported ratio away from the configured 90/10 split.
Identify the root cause in your scenario
- Open your scenario and locate the A/B test node.
- Check whether the node sits inside a loop or receives a repeated scenario trigger.
- Compare how often customers from each variant re-enter the node.
- Adjust the scenario logic so that customers from each variant re-enter the node at a similar rate, or move the A/B test node closer to your action nodes to reduce repeat pass-throughs.