If you do a lot of A/B testing in your company, congratulations! Analytics are very important when it comes to figuring out the right way to engage your audience and get them to ultimately buy whatever it is your selling. But now for some bad news – you may have been A/B testing wrong all this time.
Yesterday, I attended a presentation by Dan Chuparkoff, the VP of Product at Civis Analytics. Civis Analytics is a data and analytics giant based in Chicago. If anyone knows about testing, whether it be websites or products, it’s them.
Chuparkoff was kind enough to share his wisdom with the crowd. Fun fact: not only is he an expert when it comes to A/B testing, he is also the father of identical twins, so he basically lives A/B testing 24/7.
The take-home message from Chuparkoff’s talk was that your audience is a collection of individuals, not an homogenous group. Therefore straight A/B testing does not always work the way that marketers think it should. Different people are turned on or off by different things and standard A/B testing does not always get to the root of what turns people away, only what encourages people to click, or view, or buy.
“Your audience isn’t just one homogenous group of people, and talking to them all at the same time is generally a bad way to tackle the problem. Your audience is actually made up of lots of different people: dog lovers, cat lovers, lizard lovers; and whatever your brand might be, there are various segments floating under the covers.”
In order to make the most of your testing it’s best to fully understand the influence that your the influence your marketing is having, not necessarily the amount of clicks that it gets. It’s just as important to understand why some people didn’t click as it is to track the people who did.
“When you’re putting content out there, be conscious of the fact that some of your messages make your situation worse,” says Chuparkoff. “If all you’re doing is click tracking and trying to find the thing that most people like, you’re forgetting the fact that, of the people who didn’t click the button, some of those users are alienated,” “They’re more confused than they were, or they like your brand less.”
While it might seem complicated to try to reach every segment of your audience with the message that they will find most attractive, it is important to try and tailor your message for different segments as well as learn the segments that make up your audience and do your best to keep them engaged.
