No.4: A/B Testing “You can get with this, or you can get with that”

A/B Testing! Oh, I remember we talked about it briefly in the last lesson.

Basically it is an experimental testing method which compares various versions of the same homepage on the user’s reaction. So one can determine which improvement has the higher impact/which people like better. The one of those pages which gets favored by the users would be considered as the better model. Another good thing about A/B Testing is that it allows testing new versions of one’s page/site without compromising the business. All in all A/B Tests intend to improve the web performance for better online-service, better sale rate, higher traffic, more donations, registrations and so on while providing further valuable insights about visitor interests and demands. It is very important to note, that the focus lies on the interests and perceptions of the customer/user of the page, not the interests of any other stakeholder.

This clip from Kentico was quite helpful for me to understand what A/B Testing is in general: http://youtu.be/-Kh0xCKoNvU?t=17s Please excuse her voice pitch and the silly animations used in the entire clip. Personally I found that very annoying, good information regarding A/B Testing though and that is what finally counts!

This clip is explaining A/B Testing by using the example of a Voting Campaign and presents the benefits of A/B Testing. This is very useful for the next point on this blog I want to talk about – The Obama Re-Election Campaign in 2012!

Here, Amelia Showalter and her team did an A/B Test on various versions of fundraising mails for the before standing re-election campaign. A dozen of different mail versions were tested on one day. The difference between the best and the last successful version that day was a fundraising amount of $2,137,263, a difference you don’t want to miss when you actually start the campaign and blast the mail out to millions of subscribers.

More than the headliner of the mail, the A/B Test was also testing the amount of money Amelia and her teams would ask people to pay. As one can see, Obama is president again – would it have been the same without this A/B Test?

However, the Obama campaign didn’t get much use from commercial data like magazine subscriptions or purchase histories — the only really useful outside data was largely demographic. “I mean, the best data for us was things that we collected at the doors,” Messina told Politico. “You know, there was some commercial stuff that every campaign, including the Romney campaign, purchases. But you know, the truth is, the more we learned about data, the more we learned how important the connection was. The door-knocking, having a real conversation with people on the doors, that really mattered.” The campaign also relied heavily on information that is public record, like voting histories. None of this is functionally different from the kinds of insights that campaigns have used for years to target direct mail, and people who worry that politicians are learning how to tailor their message to their audience based on internal research are decades late to the game.

Sources used for this post:

https://www.optimizely.com/ab-testing/

https://markstaton.files.wordpress.com/2013/10/guidecontrolledexperiments.pdf

http://www.businessweek.com/printer/articles/84306-the-science-behind-those-obama-campaign-e-mails

http://techpresident.com/news/23173/obama-campaigns-legacy-listen-experiment-and-analyze-everything