Most marketers would agree that testing your online advertisements is beneficial because, as we have all experienced at one time or another, we may love our materials but our customers don’t always feel the same way. Most would also agree that testing targeted landing pages is just as beneficial. After all, it does no good to send traffic to your site only to find that your customers aren’t converting. And with so many tools out on the market, you can easily test your pay-per-click ads (PPC ads) and your targeted landing pages. All it takes is some planning and a little understanding of randomized experiments.
Split testing and A/B testing are terms that are often used interchangeably in relation to online marketing, but these two terms are not actually the same. In fact, A/B testing, and the often-ignored multivariate testing, is a method of split testing. And with so many of us strapped by meager resources, understanding the differences between these methods can help determine which test to implement to provide valuable insight into your online advertising campaign to assist in making decisions, and thus, ad efficiency.
The Digital Marketing Glossary defines split, A/B, and multivariate testing as the following:
- “Split testing is the process to deliver different versions of a webpage or website to visitors or to deliver different email versions to recipients within a same campaign. In both cases, the goal is to measure impact on conversion (e.g. registration rate or order rate on websites or open and click rate for email).”
- “A/B testing is a process used to measure the impact of a variable change in regards of a goal achievement (click, validation, web form completion, etc.).”
- “Multivariate testing (MVT) is a particular form of split testing by which several versions of several components of a webpage or website are dynamically combined to give multiple versions of the page or website. The goal is to measure how the different changes can affect conversion rate for a given objective (e.g., order rate, webform completion rate, subscription rate, etc.) and to give the winner page version.”
As The Digital Marketing Glossary points out, the A/B testing method is more specific than split testing in that A/B tests measure just one variable, whereas split testing measures many variables. These variables can include, among other things, headlines, subheadings, copy, images, call to action (CTA), CTA position, different offers, button color, copy length, and background. Multivariate tests also measure one variable but, unlike A/B tests, multivariate tests conduct many A/B tests on one webpage at the same time. As you might imagine, each method has strengths that can improve your online campaign.
Split testing, when implemented at the beginning of your campaign, can provide some direction on what concept and design resonates with your audience. The main purpose of split testing is to provide two very different versions of landing pages (or PPC ads) and continue to improve on those concepts rather than waste time on a concept that your customers don’t like.
So, for example, if you were testing a landing page (this also applies to PPC ads), you would create two different versions of your landing page with each version having notably different variables. You would then make sure that each concept has the same conversion goal (this is critical), such as requesting more information about a program. Comparing dissimilar conversions would be like comparing apples to oranges since a lead is not the same as, say, a PDF download. Then send half of your traffic to your first landing page and half of your traffic to the second landing page. A winner should emerge as to which concept your audience prefers.
Again, this is a great method to use when you are launching your online campaign and you don’t have much knowledge about with message and/or design your audience prefers.
Once you have a clear winner from the split testing, you can begin A/B testing the winning concept to find out what it is about the winning concept that does better. This can be done by selecting one variable on your landing page (again, this also applies to PPC ads) and create a new version with just a change to the variable selected, such as changing the text on your CTA. Once the winner is determined, you can start a new A/B test on another variable.
If you find that you do not have a winner, this does not mean that you should stop testing; on the contrary, you should always be testing. Try the test one more time with the same variable but a new version to see if this provides more insight. For example, if you change the text on your CTA and see no clear winner, try testing another version of text. If this still produces no winner, you might conclude that the text on the CTA is insignificant to your audience. You can then move on and try testing another variable.
As you might imagine, A/B testing can take time to test each variable individually. This method of experimenting greatly can speed up the process by testing many variables at the same time.
Multivariate testing can also reveal better combinations of variables than A/B testing would otherwise provide. For instance, A/B testing might reveal that variation one of the CTA performs better than the second and that headline one performs better than the second. However, A/B testing won’t be able to analyze if the CTA one with headline one does better than CTA two with headline two. Let me draw this out, as it gets kind of confusing:
As the figure above shows, Landing Page 4 was not tested using the A/B method. Still, even though the multivariate method tests more combinations, it does require more versions of landing pages and a lot more traffic to your site to determine a winner. The more variables you test at one time, the more versions and traffic you will need for definitive results.
Testing Ads and Landing Pages Together
You can test your PPC ads and landing pages at the same time but, as with any type of testing, you want to make sure that you hypothesize and plan your tests accordingly. The reason that planning is so crucial is because each element of your online campaign can impact the results, as they are not isolated, and you don’t want to make unfavorable changes.
The figure below shows one scenario in which you might make an adverse decision depending on how you analyze the data. In this case, traffic from many sources are being sent to one landing page, much like an admissions landing page: PPC ads are sent to this single page along with people that used the main navigation on your website. If you split test this admissions page and analyze the overall results, you might see a clear winner.
However, your PPC audience might convert better on Landing Page 2 as they landed on this page with a different mindset than someone who was browsing your site. In order to determine if this is the case, you would need to send your PPC customers to a duplicate landing page with a unique URL, as the figure below shows. This way you will know where the traffic originated and if this impacts which landing page performs better.
Above, you would not want to use Landing Page 2 for all your traffic, just your paid advertisements.
In the end, there is an endless amount of variables that you can test within your online campaign and you can get extremely complex, assuming that you have enough traffic to give you answers.
If you have just started experimenting, keep it simple. Try testing just one variable and gradually get more complex as you become more comfortable. After all, the worst that you can do is not test at all.