Free calculator

Bayesian A/B test calculator

Enter the visitors and conversions for your control and each variant to get the chance to beat control, credible intervals and the expected lift. It uses the same Bayesian method the Split Test Pro results page uses. No sign-up.

Your test data

Control

Variant A

Advanced

The default prior, Beta(1, 1), treats every conversion rate from 0% to 100% as equally plausible before any data.

Result

VersionConversion rate95% credible intervalProbability to be bestChance to beat controlUplift
Control
Variant A

Each number comes from 100,000 simulated draws per version. The draws are seeded by your inputs, so the same inputs always give the same result.

Embed this calculator

Paste this code into any page. The calculator loads with the numbers entered above and follows your visitor's light or dark setting.

How the calculator works

The calculator uses the same Bayesian method the Split Test Pro results page uses. Instead of a p-value, it answers the question product teams actually ask: how confident are we that the new variant is better?

The model: Beta-Binomial

A visitor either converts or does not, so each version's conversion rate is modeled as a Beta distribution. With the default prior, the posterior for a version is Beta(α, β), where α = conversions + 1 and β = (visitors − conversions) + 1. Each version is modeled on its own.

The default prior: Beta(1, 1)

The +1 on each parameter is the Beta(1, 1) prior: a uniform prior that says, before seeing any data, every conversion rate from 0% to 100% is equally plausible. It is the standard uninformative prior for binary A/B testing and the one the Split Test Pro results page uses. As data accumulates, the posterior tightens around the true rate. Under Advanced you can set a different prior.

What a credible interval is

A credible interval is the range of plausible values for a conversion rate, with a stated probability. A 95% credible interval means there is a 95% chance the true value is in that range, given the data. A wide interval means there is not enough data yet. A narrow one means the data has converged.

How the Monte Carlo draws work

The probabilities are computed by Monte Carlo sampling. For each version the calculator draws 100,000 random samples from its posterior, then counts:

  • Chance to beat control is the share of draws in which the variant's sample is larger than the control's.
  • Probability to be best is the share of draws in which a version's sample is the largest of all.
  • Uplift is (variant sample − control sample) / control sample × 100 for each draw. The figure shown is the median, with its credible interval underneath.

With one variant, probability to be best equals chance to beat control. With two or more, probability to be best adds up to 100% across all versions, while chance to beat control is worked out for each variant on its own. The draws are seeded by your inputs, so the same inputs always give the same numbers.

Reading the result

The sentence above the table follows the same thresholds as the Split Test Pro results page. A variant is winning when its probability to be best reaches 95%, and trending ahead from 80%. Control is winning at 5% or below, and trending ahead at 20% or below. Until every version has at least 10 conversions the result reads Not enough data yet, whatever the probabilities say.

An uplift interval entirely above zero is a strong winner. One that straddles zero is uncertain and needs more data.

For the full explanation of the model, read our guide to Bayesian A/B testing.

Frequently asked questions

  • What is a Bayesian A/B test calculator?
    It turns the visitors and conversions of an A/B test into probabilities you can act on: the chance that each variant beats control, the probability that each version is the best, and a credible interval for every conversion rate and for the uplift. This one uses the same Bayesian method the Split Test Pro results page uses.
  • What does chance to beat control mean?
    It is the probability, given your data, that the true conversion rate of a variant is higher than that of the control. A 97% chance to beat control means the variant came out ahead of control in 97% of the simulated draws.
  • How is a credible interval different from a confidence interval?
    A credible interval makes a direct claim: there is a 95% chance the true value is in this range, given the data. A confidence interval makes a claim about the procedure: if you ran the experiment 100 times, 95 of the intervals would contain the true value.
  • Is 95% the right threshold?
    95% is a convention, not a mathematical truth. 90% is fine for low-stakes tests such as cosmetic changes. 99% suits high-stakes tests, such as changes to checkout or pricing, and takes longer to reach.
  • When should I keep the test running?
    Keep it running while the result is uncertain, and until every version has at least 10 conversions. A pragmatic rule is to wait for a 95% probability, at least 300 to 500 visitors per version and at least 7 days.
  • What prior does the calculator use?
    Beta(1, 1) by default: a uniform prior that treats every conversion rate from 0% to 100% as equally plausible before any data. You can set a different prior under Advanced.
  • Why do I get the same numbers every time?
    The calculator draws 100,000 random samples per version, and the random numbers are seeded by your inputs. The same inputs, or the same shared link, always give the same result.
  • Is the calculator free?
    Yes. The calculator is free to use, needs no sign-up and runs in your browser.

Run better tests.
Win more often.

Start your free trial and see the difference.

  • No credit card
  • 14-day free trial