It is important to understand that the recommendation engine always generates a ranked list of item IDs before applying the catalog filters. For metric-based models, this ranking is driven by the configured metric (for example, views or purchases); for AI models, it is driven by the learned model logic.
Because the catalog filters are applied after ranking, the Test tab can appear to show “random” items even when the engine has valid results. In these cases, many of the highest-ranked items may be filtered out, and what you see in the UI is whatever still passes the catalog filters, which can look random at a glance.
If you use the following Jinja in an email template, then any missing items (after catalog filtering) are backfilled with random catalog items to reach the requested size:
{% set reco = recommendations("recommendationID", 100, fill_with_random=true) %}
{{ reco }}As a result, many of the first items in the reco list can be random fillers, and the true engine results may appear only later in the array.
For debugging, it is better to inspect the “pure” engine output without random backfilling. In such cases, use:
{% set reco = recommendations("recommendationID", 100, fill_with_random=false) %}
{{ reco }}This disables the random fill, so the items you see in reco are strictly the engine’s own results after catalog filters. If the list is short or empty, that usually points to overly strict filters or a data alignment issue (for example, item IDs not matching between events and catalog).