What is answer engine optimisation (AEO)?
Learn what answer engine optimisation means and how to make useful business content easier for search and AI systems to extract and cite.
Marketing glossary
What is A/B testing?
A/B testing replaces “I prefer this version” with a controlled comparison of behaviour. It can improve marketing decisions, although small samples and casual methods often produce certainty the evidence has not earned.
What A/B testing means
A sound test starts with a hypothesis, such as: “Making delivery timing explicit will increase completed quote requests because it removes uncertainty.” Suitable visitors are randomly assigned to version A or B at the same time. The business chooses a primary metric in advance and analyses whether the observed difference is sufficiently reliable and commercially worthwhile.
Randomisation matters. Sending version A on Monday and version B on Friday is a before-and-after comparison, not a clean A/B test, because audience mix, demand and countless outside factors may differ.
A practical SME example
A training provider wants more suitable visitors to request a team quote. Its current landing page uses a button labelled “Submit”. Version B changes the label to “Request a team quote”, leaving the rest of the page unchanged.
The provider splits eligible traffic evenly and measures completed qualified requests, not button clicks alone. If version B generates more clicks but no additional completed forms, the new wording improved an intermediate action without improving the actual objective. The business should not declare victory based on the friendliest metric.
Why A/B testing matters
Controlled experiments can:
- distinguish a causal improvement from ordinary fluctuation;
- resolve disagreements without relying solely on seniority or taste;
- quantify the effect of messaging, layout or journey changes;
- build knowledge about what a specific audience values;
- protect a strong existing experience from an untested redesign.
Tests can cover adverts, emails, forms, pricing presentation and onboarding flows. They work best where sufficient comparable traffic and a clearly measured outcome exist.
Common mistakes
The first mistake is stopping as soon as one version moves ahead. Repeated checking increases the chance of selecting a random fluctuation unless the method accounts for sequential decisions. Set the duration, sample requirement and decision rule before starting, and cover relevant business cycles.
Another mistake is changing several unrelated elements at once. If the headline, image, offer and form all change, you learn which package won but not why. That may still support a practical decision, but it cannot establish which individual change caused the result.
Businesses also run underpowered tests on tiny traffic volumes. In that case, customer interviews, usability sessions or a larger strategic change may produce better evidence. A non-significant result does not prove the versions are identical; it may mean the test could not detect a useful difference.
Finally, assess guardrail metrics. A variant that raises conversion rate while reducing lead quality or increasing cancellations can damage the underlying business.
Related guides
Related terms
Is split testing the same as A/B testing?
The terms are commonly used interchangeably. A/B testing usually means comparing two concurrently randomised versions, while “split test” is sometimes used more loosely for any divided traffic comparison.
Can an SME run A/B tests with low website traffic?
It can, but detecting modest effects may take impractically long. Prioritise larger, well-founded changes and higher-volume points such as emails, or use qualitative research when a controlled test cannot reach an informative sample.
Who wrote this
Steve Ward.
Steve founded Epitomise in 2017 after UK, international and global marketing leadership roles, most recently as Global CMO of the Vitec Group’s Videocom Division. He works with SME and technology businesses on strategy, positioning and the execution that follows — more about Steve.
Read next
More on Marketing glossary and beyond.
What is bounce rate?
Learn what bounce rate means, why analytics tools calculate it differently and how to diagnose a high bounce rate without guessing.
What is brand positioning?
Learn what brand positioning means, how to define it and how an SME can occupy a useful, distinctive place in customers' minds.
Want a second opinion on your marketing?
A short discovery call is usually enough to tell. No deck, no pitch.
Book a discovery call