How to split-test a website with PHP
by Chris Woodford. Last updated: September 19, 2022.
You've shelled out for the best web design company you could afford. You've paid big bucks to the
SEO company to shoot you up the rankings. But if your site's designed
to make you money—through product sales or advertisements—there's
one more thing you must do: optimize it with split-testing.
Most people set their sites up in an arbitrary way on the basis of
what looks nice or "seems to work." But as web pros know only too
well, even apparently insignificant changes to your site's structure,
layout, colors, or type can bring big payoffs in click-throughs and
orders. Split-testing (sometimes called A/B testing) is a simple way
of optimizing your website by comparing the results you get with
two different versions of the same page. Let's take a closer look!
Photo: Split-testing web pages is the best way to determine which of various different site layouts gives you the results you want—in sales or advertising revenue. All you need to master is a tiny bit of PHP, like this.
Why simple testing doesn't always work
The simplest way to test the effectiveness of a web page is to alter your current design,
try the new design, and compare the results. Sounds simple—and it
is. The only trouble is, it's too simple. If you run design A
for a whole week and then run design B for the whole of the next week
and compare the results, you won't necessarily establish which design
is the best. If you're tuned in to how your web traffic fluctuates
from day to day and week to week, you'll know only too well that many
things can change from one week to the next. You might hit a school holiday,
for example, or a run of really good weather that sees large numbers
of people quitting their desks for the beach. Similarly, if you're
trying to compare advertising performance, you probably know that
many factors can change week by week, day by day, and even hour by hour. There may be more
people advertising in the second week than the first, producing a
bigger advertising inventory, higher paying ads, and better
results—irrespective of which design is running on your site.
Someone might flag your site with a social bookmark or mention you on
a forum, sending you a sudden, massive batch of very dubious quality
traffic that dramatically reduces your site's performance.
All told, running one design for a day, week, or month before alternating with another
design is going to give only a crude idea of what works best. You
could keep switching back and forth between different designs and use
some form of graphical or statistical analysis package to compare
your results each time you switch but, the trouble is, it's going to cost
you money to do this! One of your designs is almost certainly a bit
better than the other. If you change your original layout for the
worse, every day you run a less-effective, alternate design is a day
that dips into your dollars. That's why you need split-testing.
What is split testing?
Split testing dates back to early "key-code" tests of newspaper advertising devised by
Claude Hopkins
about 100 years ago. He printed slightly different versions of an advertisement (perhaps one with one headline, one with another), including a coupon with a return address that featured a different mailing box number. By comparing the number of people who replied to reach coupon, he could reliably measure the effectiveness of each ad variant. Testing multiple
variants over days or weeks, he would eventually arrive at an optimized ad. Today's split
testing of online ads is just an updated, hi-tech version of the Hopkins method.
The method is simple. You create two different versions of a web
page and arrange for them to be shown randomly, alternately, to your
visitors. You track the performance of each page separately and
compare your results from each page after a decent number of
impressions (a few thousand is usually enough but a big difference in
performance will show up after only a few hundred).
You can do split testing either with javascript (so the two variants you're comparing
are included in one HTML page and the user's browser creates a single
page on the fly) or with PHP (where one page or the other is built
on-the-fly by your web server). Google's Adsense blog has a really
simple example of how to do A/B javascript testing—in this case,
it's being used to optimize the colors of advertisements. I much
prefer using PHP. Even if you have no experience of using it, it's
simpler to understand and completely transparent to search engines,
which "see" either one complete variant of your page or the other
(not some complex, glued together hybrid as with javascript). Using
PHP also means you don't have to tinker with any of the advertising
code (which is typically prohibited) and you can test much
bigger differences between pages. For that reason, I won't discuss
javascript testing any further here.
How to do split testing with PHP
PHP, if you don't already know, is a superb program that runs at the web server end of things, building web pages on the fly before they're sent out to a
browser. It's immensely powerful and you can spend ages learning how
to do all kinds of neat things with it. But, even if you have no
experience of using PHP, you can learn just enough to run simple web
page tests in about five minutes. Here's how!
1. Make sure PHP is running on your server and enabled with .htaccess
If you're using free, low-budget, or Windows web hosting, you probably don't have PHP.
Check with your hosting company first. Once you've confirmed this,
ensure PHP is working for your site and that files with the suffix HTM (or HTML) are correctly processed
by the PHP processor. For Apache servers, you will probably need to include something like the following
line in your .htaccess file:
AddType application/x-httpd-php .htm .html
Exactly what you need in .htaccess will vary according to how your server is set up. On another server
I run, I have to use a slightly different variant and explicitly say that I'm using a certain version of PHP:
AddType application/x-httpd-php5 .html
AddHandler application/x-httpd-php5 .html
2. Save your original web page (or site)
You'll be comparing a new version of your web page against your original and possibly
modifying the original quite a bit, so do the sensible thing and make
a backup of the original to start with.
3. Create a new version of your web page
Now alter your page to create the new, "improved" version you want to test. Maybe you
want to test a different ad layout, change the colors, use a
different style sheet, rewrite some wording, place a big "Place
your order now" button at the very top, or whatever. You can
compare versions that are as different as you like, but ideally you
want to change only one thing at a time so you can be sure
which change made the difference. So maybe change the colors first
and test those. Then, when you've found the best color scheme, change
the layout. Then change the wording. Then add your "order now
button." And so on. Make each change and evaluate it separately
before monkeying with anything else. If you change everything all at
once, you won't know where you are: you won't know what improved or
impaired the response to your page.
4. Create two small pages based on the differences
Chances are the two versions of your web page will differ only slightly. Suppose you want
to compare how many click-throughs you get with black text on a white
background versus using white text on a black background. You can do
that by using two slightly different style sheets so, in this case,
the two versions of your web page will differ only in that they need
to load different stylesheet files. Similarly if you're assessing the
effect of adding a "Place your order now" button, one version of
your page will have an extra line or two linking to the image and
order page that the other version doesn't have. So now create two
small HTML pages including only the code that is different. For
example, the first file might be like this. Call it file1.html:
<P>
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
</P>
and the second file (including the order now link) might be like this. Call it
file2.html:
<P>
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
ordinary text ordinary text ordinary text ordinary text
</P>
<P><A HREF="ordertest.html">Order now!!</A></P>
5. Add PHP code to your original web page
Now you need to edit your original web page and remove the section you've identified
that's going to alternate between the code in file1.html and the code
in file2.html. Cut that section out entirely and replace it with this
little bit of PHP code:
<?php
srand((double)microtime()*1000000);
if (rand(1,2)==1)
{
include("file1.html");
}
else
{
include("file2.html");
};
?>
See how simple that is? It will randomly include the bit of HTML from either file1.html or file2.html in your web page
each time the web server is requested to send the page out to a
browser. The browser will receive a page including either the file1
or file2 code, but (unlike in a javascript test) there will be no
clue that the page has been built with PHP on the fly.
6. Set up a way to count the results
In this simple example, you could monitor effectiveness simply by seeing how many
orders you get through the page ordertest.html compared to your usual
ordering page. You could see how many times each page is loaded and
how many orders you get as a result. If you want to compare the
performance of two different advertisements, you can put one lump of
advertising code in file1, another lump in file 2, and set up
different channels/counters to monitor the CTR from each.
7. Monitor the results
Now all you have to do is wait for the results to come in. There are still some things to be
wary of. The first is that if there is no real difference between the
two pages you're testing, any counters or measurements you set up
will likely flip back and forth showing first one page in the lead
and then the second page, with the position constantly alternating.
If that happens, it usually means the pages are just as good as each
other, even if one page ultimately gets slightly better results than
the other. If one page is constantly ahead of the other page
throughout your test, it's obviously a better page. But do be sure to
allow enough page impressions to give a reliable result (certainly a
few hundred and maybe a few thousand, if you have time). Don't be
tempted to draw conclusions from only a few impressions.
If you want to be really certain (and scientific), run a
chi-square test
and check for statistical significance.
8. Don't stop there!
Once you've established one change that works, go back and incorporate it into
your entire site and monitor the results. Then work on identifying
another change and see if you can improve things a bit more. Keep
doing this and, over time, you should systematically and
significantly improve your site's performance. Every so often, run a
test where you compare your current version of the site with an
earlier incarnation to reassure yourself that the cumulative changes
you've made are all working to your advantage. That's just a little
double-check for your own peace of mind!
