What is A/B testing

Learn how A/B testing helps optimize your website by comparing different versions to improve performance, integrated seamlessly with Umbraco Engage.

It goes too far to give a total overview of what A/B testing is. There are some great resource out there that can inspire you and can go in real depth what A/B testing is.

For now we will stick with this explaination: A/B testing is a methodical way of testing (and hopefully improving) your website. By serving multiple versions of a webpage, content block, or website to different visitors, gathering data on which version performs better for a specific goal.

Why should you A/B test your website?

Your website is always underperforming its potential. Your website will never have a conversion rate of 100% and probably not even near that conversion rate. All visitors do exactly what you want them to do. By testing specific parts of your website you can experiment and hopefully increase your conversion rate. The internet gives you the ability to test with different variants of your website and see what happens.

A/B Testing in Umbraco

There are already a lot of great A/B testing tooling out there (Google Experiments, Optimizely, Visual Website Optimizer). They do a really great job on A/B testing but they lack a key feature that Umbraco Engage offers: full integration within Umbraco. This allows you to integrate A/B testing in your daily workflow instead of an add-on on top of your website. This means:

• You can A/B test your content in the same user-friendly way as Umbraco does, allowing you and your editors to start A/B testing.

• The A/B test is rendered automatically without any additional line of code. This means that you don't have to reserve time to "implement A/B testing". It is there and you can start testing immediately. Also that means there is no content flickering like in many other tools.

• You can use all the resources that Umbraco gives you. You can reuse images of the media library, you can pick contentblock as you are used, and you do not have to worry about changing URL's while doing a test.

• Within Umbraco Engage you will see that A/B tests are running, so your fellow editors are aware of this and will not ruin your experiments

• Because we use first-party analytics and cookies you will track data across all your visitors and not only on a subset of customers which do not have any ad- and cookie block tooling

• Our A/B test is context aware and will make sure that the A/B test is render everywhere correctly. For example; if you are testing a catchy product title it will automatically not render only that new title on the page itself, but also on your overview, your homepage, your shopping cart or at any other place you're using the title. This is impossible with any other tool!

Some great resources

Many books explain the importance of A/B testing and why you should start testing your website today. A short list of the books that inspired the Umbraco Engage team:

• Dan Siroker and Pete Koomen. A/B Testing. The most powerful way to turn clicks into customers. Hoboken, New Jersey: John Wiley & Sons, Inc., 2013. Print

• Chris Goward. You should test that! Indianapolis, Indiana: John Wiley & Sons, Inc., 2013. Print

• Will Kurt. Bayesian statistics the fun way. San Fransisco, William Pollock, 2019. Print