How long should you let your conversion optimization experiments run on average?

There is no average when talking about the length of A/B conversion tests. A lot of conversion rate optimization services say that it's possible to do ab tests with low traffic sites but the truth is, that data may be misleading or the length of the test to get accurate measurements can be months if not years.

Depending on what are the steps of conversion optimization you took, the average time for the test can be calculated with these numbers:

• Existing conversion rate (estimate)
• Minimum conversion rate increase you want to see
• Number of variations or combinations towards control variation (the more you have the more visitors/time you'll need)
• The average number of daily visitors to your page you are testing
• Percent of visitors included in the test? (50/50% split of visitors is suggested)

The easiest way to find out is to use an A/B Test Duration Calculator from WVO where you input these numbers and a formula calculates them and shows the results.

You will get the “Total number of days to run the test” but be careful as this is an estimate only.

A good rule of thumb is to have 250 conversions per week for 2 weeks for each variation, to get data that is more accurate, but even then it will be a probability only and not a rule. Even if you have a 100% statistical significance you can't be 100% sure but you can be pretty confident.

Also, take into consideration that a conversion increase of 5% needs way more traffic and time to get to that statistical significance than if you had a 90% lift. Small differences could just be chance but large ones are the real deal.

Also a/b testing can be different for other fields or terms for conversion optimization.

Conversion optimization and your average dice throw you should never forget

Think about having a game of dice. If you throw 5 times and you get 5 sixes, doesn't mean you'll always have 5 sixes when you throw and the possibility to repeat that is quite low. So the “element of chance” needs to be considered when a/b testing.

If you throw the dice 1000 times you probably will not get the 16.6% of ones, twos, threes etc. but it will get close to it.

The short answer to How long should you let your conversion optimization experiments run (on average) is, therefore… it depends.