Chicwish
A new-arrival rule that was quietly costing 5% of revenue.
The store
CHICWISH is a fashion brand selling to shoppers in the US and Europe, running on Magento. Two years before we started working together, the team set out to fix a common problem: new arrivals weren't getting enough impressions. They turned on a new-arrival rule in their previous platform that placed new products in every even-numbered slot on every category page, so new arrivals would get more impressions. Almost every merchandising tool has a setting like this, and it is close to the industry's standard fix.
Before the test
Nobody had a reason to doubt it. The rule never threw an error and never showed up in a report. In the admin panel, it was just a toggle that had been on for two years. And it seemed to be working: new arrivals were getting more impressions than before, and their sales had gone up. As far as the team could tell, the problem was solved.
How we tested
We ran a 50/50 A/B test for 35 days, with revenue as the primary metric. We also tracked new-arrival impressions and the share of new arrivals that sold at least one unit. The only difference between the two groups was how new arrivals were shown:
Group A kept the new-arrival rules set up in their previous merchandising platform.
Group B let Bytem AI's model decide, shopper by shopper, which new arrivals to show, when, and where on the page.
What we found
The old rule did deliver the impressions it promised, but it cost at least 5% of revenue.
The reason is simple: it pushed new arrivals in front of every shopper, whether they liked them or not. Shoppers who liked a new style would have found it anyway. For everyone else, the new arrival took a spot that could have held something they would actually buy.
"For two years we boosted new arrivals on our category pages, because our old platform had a setting for it. One A/B test with Bytem showed it was costing us at least 5% in revenue. We had been pushing new products at everyone, whether they wanted them or not."
A setting that never shows up in a report can still show up in your revenue.