You’ve Probably Been A/B Testing Wrong All Along

If you do a lot of A/B testing in your company, congratulations! Analytics are very important when it comes to figuring out the right way to engage your audience and get them to ultimately buy whatever it is your selling. But now for some bad news – you may have been A/B testing wrong all this time.

Yesterday, I attended a presentation by Dan Chuparkoff, the VP of Product at Civis Analytics. Civis Analytics is a data and analytics giant based in Chicago. If anyone knows about testing, whether it be websites or products, it’s them.

Chuparkoff was kind enough to share his wisdom with the crowd. Fun fact: not only is he an expert when it comes to A/B testing, he is also the father of identical twins, so he basically lives A/B testing 24/7.

The take-home message from Chuparkoff’s talk was that your audience is a collection of individuals, not an homogenous group. Therefore straight A/B testing does not always work the way that marketers think it should. Different people are turned on or off by different things and standard A/B testing does not always get to the root of what turns people away, only what encourages people to click, or view, or buy.

“Your audience isn’t just one homogenous group of people, and talking to them all at the same time is generally a bad way to tackle the problem. Your audience is actually made up of lots of different people: dog lovers, cat lovers, lizard lovers; and whatever your brand might be, there are various segments floating under the covers.”

In order to make the most of your testing it’s best to fully understand the influence that your the influence your marketing is having, not necessarily the amount of clicks that it gets. It’s just as important to understand why some people didn’t click as it is to track the people who did.

“When you’re putting content out there, be conscious of the fact that some of your messages make your situation worse,” says Chuparkoff. “If all you’re doing is click tracking and trying to find the thing that most people like, you’re forgetting the fact that, of the people who didn’t click the button, some of those users are alienated,” “They’re more confused than they were, or they like your brand less.”

While it might seem complicated to try to reach every segment of your audience with the message that they will find most attractive, it is important to try and tailor your message for different segments as well as learn the segments that make up your audience and do your best to keep them engaged.

Did you find this article helpful? Click on one of the following buttons
We're so happy you liked! Get more delivered to your inbox just like it.

We're sorry this article didn't help you today – we welcome feedback, so if there's any way you feel we could improve our content, please email us at

Written by:
Kristin is an aspiring entrepreneur who is enthusiastically navigating her way through the DC startup space. She has an unending passion for learning and is never satisfied with the status quo. During the day she is an ops, biz dev, and marketing maven for Fission Strategy
Explore More See all news
Back to top
close Building a Website? We've tested and rated Wix as the best website builder you can choose – try it yourself for free Try Wix today