Foldwren

A/B testing for small websites: is it worth it?

Why split tests give unreliable answers on low-traffic websites, and the simpler methods that tell a small business what to improve.

A/B testing means showing half your visitors one version of a page and the other half a slightly different version, then seeing which one produces more enquiries or sales. Large online retailers do it constantly. For most small business websites, our honest view is that it is not worth the trouble, and the same effort is better spent elsewhere.

That is not because testing is a bad idea. It is because the maths needs more visitors than a local business usually has.

How a test is supposed to work

You create two versions of a page. Version A has the current headline. Version B has a new one. A testing tool splits your visitors at random, counts how many in each group go on to enquire, and declares a winner.

The logic is sound. Because the only difference between the groups is the thing you changed, any difference in results should come from that change. But that only holds when enough people pass through each version for luck to even out.

Why low traffic makes the answer unreliable

Flip a coin ten times and you might get seven heads. You would not conclude the coin is rigged. Flip it ten thousand times and get seven thousand heads, and you would be right to be suspicious. Small samples throw up lopsided results by chance all the time.

Now apply that to a website. Say you are an accountant whose site gets 400 visits a month and 12 enquiries. Split the traffic and each version sees 200 visitors and about six enquiries. If version A ends the month on five and version B on eight, B looks like a clear winner.

It is not clear at all. A difference of three enquiries could come from a referral who happened to land on B, a week of school holidays, or plain luck. Run the same test again next month and the result could easily reverse.

To be confident, you need a good number of conversions on each version, and on a small site that can mean running a single test for many months. In that time the season changes, you run a promotion, a competitor closes. By the end you are no longer comparing like with like.

The real danger is false confidence. Changing a headline on a hunch is fine. Believing you have proven something, and building later decisions on it, is where small tests lead people astray.

When testing can be worth doing

There are a few cases where a small business has enough volume for testing to mean something:

  • An online shop with a steady, high number of orders every week
  • Paid ads, where the ad platform shows different headlines to a large number of people and reports which gets more clicks
  • Email subject lines, if your list is large enough that each half is still a big group

If none of those describe you, skip formal A/B testing with a clear conscience.

What to do instead

You can still improve your site methodically. These approaches work at any traffic level.

Fix the obvious without a test

You do not need an experiment to know that a missing phone number, a slow page or a twelve-field form is costing you enquiries. Well-established good practice does not need to be proven again on your site.

Watch real people use it

Ask three or four people who do not work in your industry to complete a task on their own phone while you watch in silence. "Find out whether we service your suburb and ask for a quote" is a good one. Twenty minutes of this will teach you more than a three-month split test. You will see where they hesitate, what they miss and which words confuse them.

Ask your customers

When someone books, ask what nearly stopped them, or whether there was anything they could not find on the website. People are usually happy to tell you, and the same two or three answers come up again and again.

Compare before and after

Make one meaningful change, write down the date, and compare the following few months with the same period before, or with the same season last year. This is not scientific, so treat the result as a hint and not as proof. Bigger changes show up more clearly than small ones. Rewriting your service page so it answers real questions will move the number in a way that a new button colour never will.

Look at behaviour

Analytics can show which pages people leave from, how far down they scroll and whether they tap your phone number. If most visitors leave your pricing page without doing anything, that page deserves attention before any other.

A routine that suits a small site

  1. Once a quarter, go through your site on a phone as if you were a customer.
  2. Watch one or two people attempt a simple task.
  3. List the problems you found and rank them by how obvious and how easy they are to fix.
  4. Fix the top one or two, and note the date and what you changed.
  5. Check enquiries over the next few months and see whether things have shifted.

Put your energy into bold, sensible improvements and into attracting more of the right visitors. If your traffic grows to the point where each version of a page would see plenty of conversions in a few weeks, testing becomes worth a second look. Until then, common sense and a few honest observers will serve you better.

Your order