A redesign and CRO testing are not competing answers to the same question; they solve different problems. The default for a store that basically works is testing, because tests are measurable and low-risk, and a full redesign should be reserved for specific conditions. Nielsen Norman Group's guidance is exactly this: prefer incremental change, and reserve radical redesign for when incremental gains have run out (NN/g, 2015).
This is the framework for deciding which your store needs, and how to sequence them so you don't gamble revenue on a guess. It sits inside the complete Shopify CRO guide and connects to the diagnostic in why your CVR is stuck.
I run Skuology and build Upsellr. This comes from 80+ Shopify projects and over $100M in combined eCommerce revenue.
Key Takeaways
- Redesign and CRO tests solve different problems; default to tests, redesign only when specific criteria are met.
- Test wins are mostly incremental: 60% deliver under 20% lift (Convert, 2026), but they are measurable and compound.
- Redesigns are big, unmeasurable swings with diminishing returns: average UX gain fell from ~247% to ~75% (NN/g, 2020).
- Redesign when tech is outdated, architecture is incoherent, the brand shifted, or conversion sits far below the ~1.4% baseline (Littledata, 2023).
- The right move is usually redesign on a tested foundation, then keep testing; never use a redesign to escape measurement.
What a CRO test is good at
A CRO test changes one thing, measures the effect against a control, and keeps the result only if it wins. That discipline is the whole value: you know what worked and by how much. Tests are also rigorous in practice. On Convert's experiment database, roughly 70% of tests reach 95% or higher statistical confidence (Convert, 2026), so the wins you keep are real, not noise.
The honest limitation is that wins are mostly small. 60% of A/B tests deliver under 20% lift, and 84% come in under 50% (Convert, 2026). That sounds underwhelming until you see it compound: a store that banks a string of measured 5-to-15% wins pulls ahead steadily, and it always knows which changes did the work.
Testing is the right tool when you have enough traffic to reach significance, the store basically functions, and you're chasing the next increment with attribution you can trust. For most Shopify stores past the basics, that's the situation they're actually in.
What a redesign is good at
A redesign changes many things at once: layout, hierarchy, navigation, brand. That's its strength when the current store is genuinely broken or dated, because no single test can fix an incoherent foundation. A redesign can reset the whole experience in one move in a way incremental testing never will.
It's also its weakness. Because everything changes together, you can't attribute the outcome; you get a new number, not an explanation. And redesigns face diminishing returns. Nielsen Norman Group found that the average improvement from a UX redesign shrank from around 247% in 2006 to 2008 down to about 75% by 2020 (NN/g, 2020), as the easy wins got used up across the industry. A redesign on a mature store is a big, unmeasurable swing.
So a redesign earns its risk only when the store needs a reset, not a refinement. The question is which situation you're actually in.
The trade-off, side by side
| Dimension | CRO tests | Full redesign |
|---|---|---|
| Risk | Low, reversible | High, all at once |
| Measurability | Clean attribution | None, everything changes |
| Speed to result | Days to weeks per test | Weeks to months |
| Typical upside | Incremental, compounding | Large but unpredictable |
| Best when | Store works, chasing increments | Store is dated or incoherent |
The table is the decision in miniature. If you value knowing why your number moved, tests win. If the foundation itself is the problem, a redesign is the only thing that addresses it.
When to run CRO tests
Choose testing when the store works and you want the next increment with proof. If you have the traffic to reach significance, a functioning checkout and product pages, and a conversion rate that's in the normal range, testing is almost always the right call. The typical Shopify store converts around 1.4%, with the top 20% above 3.2% (Littledata, 2023), so if you're near or above average, your gains live in refinement, not reinvention.
Testing is also right when you need attribution for a decision. If stakeholders want to know what worked, only a controlled test gives a defensible answer. A redesign gives you a new number and an argument about why it changed.
The targeted version of this is high value: Baymard estimates the average large ecommerce site can gain around 35% in conversion through better checkout design alone (Baymard, 2025). That's a focused CRO program, not a full rebuild.
When to redesign
Redesign when the foundation, not the details, is the problem. NN/g names the conditions: incremental gains have hit diminishing returns after years of testing, the technology or theme is outdated, the information architecture is incoherent, or the brand has genuinely shifted (2015). Add one more from the numbers: conversion sitting far below your category baseline, not a few points but a structural gap, signals the store needs a reset rather than a tweak.
The tell is that you've run out of tests worth running. If your backlog of hypotheses is empty because the current structure can't express a better idea, that's the diminishing-returns signal NN/g describes. At that point a redesign opens new ground that testing can then refine.
What doesn't justify a redesign is boredom, a new stakeholder's taste, or the feeling that the store looks dated when the numbers are fine. Those are the redesigns that quietly cost conversion.
The trap, and the Baseline answer
The common mistake is redesigning to avoid the discipline of testing. A rebuild feels like progress and skips the slow work of forming hypotheses and waiting for significance. But it throws away the scoreboard: after a full redesign you can't say what helped, so you've traded a measurable system for a guess that happened to ship. That's the redesign that doesn't move metrics, and it's avoidable.
The Baseline Framework sequences it the other way. When a redesign is genuinely warranted, build it on what testing already proved, carry the winning patterns forward, treat the new design as a hypothesis rather than a verdict, and resume testing on top of it. You get the reset a redesign provides and keep the measurement that makes every future change accountable.
That's the real answer to "which one": rarely either alone. It's tests by default, a redesign when the criteria are met, and testing again on the new foundation.
Shopify redesign vs CRO tests: FAQ
Should I redesign my Shopify store or run CRO tests?
Default to CRO tests. They are measurable, lower-risk, and capture compounding gains on a store that basically works. Redesign only when specific criteria are met: outdated tech, incoherent architecture, a brand shift, or conversion stuck far below the roughly 1.4% Shopify average (Littledata, 2023). Most stores need testing, not a rebuild.
Are A/B test wins usually small?
Yes, and that is the point. 60% of A/B tests deliver under 20% lift and 84% under 50% (Convert, 2026). The value is that the wins are real, measurable, and compound over time. A store that banks incremental wins reliably beats one that gambles everything on a single redesign with no way to attribute the result.
Why are redesigns risky for conversion?
A full redesign changes everything at once, so you cannot attribute what helped or hurt. You also face diminishing returns: average UX-redesign improvement shrank from about 247% in 2006-2008 to 75% by 2020 (NN/g). On a mature store, a redesign is a big, unmeasurable swing where a tested change would have been safer.
When does a Shopify redesign actually make sense?
When incremental testing has hit diminishing returns after years, the theme or tech is outdated, the information architecture is incoherent, the brand has genuinely shifted, or conversion sits far below your category baseline. Those are the conditions Nielsen Norman Group names for a radical redesign over incremental change (2015). Otherwise, keep testing.
Can I redesign and still keep CRO discipline?
Yes, and you should. Redesign on a tested foundation: carry forward what testing already proved works, treat the new design as a hypothesis, and resume testing on top of it. The mistake is using a redesign to escape the discipline of measurement, which throws away the scoreboard that tells you whether anything improved.
What to do next
No guaranteed lift. Whether you need a test program or a redesign depends on your store, your traffic, and where your conversion sits today. What I can promise is that the wrong choice is expensive, and most stores reach for a redesign when a test program would have served them better.
The Baseline Framework is how I decide which your store needs and sequence them so you never lose the scoreboard. If you want a straight answer on whether to test or rebuild, book a free 30-minute call and I'll walk through your numbers and which path fits.
