A/B testing. Just because you can…
Growing up my Dad had a few choice catchphrases — mostly ones I can’t write here. “Life is like a big wheel, what goes around comes around,” he’d say at just the moment the young version of me would want to retaliate to someone who did me wrong at school or when I questioned violence on the news. He particularly surprised me when his business partner orchestrated a way to virtually steal my father’s entire company just before he retired. His response? “He must need the money more than me. What goes around, comes around.” Eventually the scum got his comeuppance.
Another phrase that stuck with me from my early years was just as simple, “Just because you can, doesn’t mean you should.”
The rule in our house was that your hair was your own — and so was your haircut — because you were the one who had to live with it. So in the eighth grade I shaved the Batman symbol into the back of my head. Just because you can, doesn’t mean you should. I learned my lesson.
In today’s world of commerce, especially that which occurs online, the world literally is your oyster. There are individuals and companies willing to take both you and your company’s hard earned revenue and spend it on plugins, upsells, new websites, ecommerce solutions, advertisements and more.
Just because you can, doesn’t mean you should.
When it comes to A/B testing, and especially with a platform like Optimizely, you can virtually change anything you desire on your web pages and serve your visitors seemingly endless variations. In fact, you can choose to alter your site’s pages so as to completely circumvent your designers, marketing department and even the developers who spent their time making sure site visitors interact in certain, predetermined ways. In an instant you can undermine and eliminate these experts. And that is dangerous.
For example, say I wanted to test my BUY NOW button on my home page’s carousel. Here’s my original:
My current button does its job (passes the item ID into my cart), but I have a gut feeling it could be better positioned. Maybe if I change the color slightly the button would be both more noticeable and more click-worthy, resulting in higher conversion. This is my hypothesis, anyway. With Optimizely, the test can be up and running quickly. So, I take my URL, load it into Optimzely, and start playing with the color. Genius!
My first attempt is mundane and I change the image from a background image (with that shiny button look) to a flat hexadecimal color. I choose red because I think it feels arresting and looks good with the grey LEARN MORE button to it’s left:
But I want a better variation. One that screams “Click me! I’m a button!” So I tweak it even further:
That’s it! I love it! I make it green and now it really stands out. “Hmm,” I think, “I’m not half-bad at this designing thing. The designers should have made it green in the first place! I’m going to change more…” So I alter the headline color, too:
“Wow, perfect,” I think. And my results in Optimizely say just as much: it works amazingly well. My engagement goes up! More people add to the cart! More convert! I must be doing something right! This is when your brand manager comes over and smacks you upside your head (or fires you).
Turns out, however, our company spent thousands of dollars (and just as many man hours) choosing colors that reflect the brand image. The colors that previously existed on the page (my “Original” tab in Optimizely) were in sync with not just our business cards and letterhead, but our packaging, in-store signage, mailers, tee-shirts, company softball uniforms and even the front door of our building.
Sure, we converted a bit more in the short term, but in the long term (or quite possibly sooner) I caused the brand irreparable harm. In essence, I changed the UPS “brown” to a lighter tan; I changed the Coke logo to yellow; the Starbucks logo to blue. This breaks the emotional connection consumers have to these brands. To mess with these brand images are messing with sacred, hallowed grounds. And you just dug up the proverbial grave and spat right into it.
Just because you can, doesn’t mean you should.
A better test would been one that not only has the blessing of your ecommerce, marketing and brand teams, but one that stays within the boundaries of your company’s look and feel. Let’s try this again. Again, first the original copy block:
If you’re confident that this needs to change (or confident in your hypothesis), try to alter the presence of the elements without altering the elements themselves. What if we merely swap the LEARN MORE and BUY NOW? What would we learn then?
What if we test omitting the LEARN MORE button altogether?
Or changing the copy in the BUY NOW button to “ADD TO CART”?
Now you have four variations: Original, Variation 1 (swapped buttons), Variation 2 (omitting one), Variation 3 (omitting one and changing type). With multivariant testing you can also change the headlines in conjunction with this test to find the best combo of header + button combo. (First, be sure to speak with your copywriter! Saying “100% Guaranteed!” may get more clicks, but it may not be true, cause complaints, and could result in legal action.)
It’s important to remember that in A/B testing (or rather sometimes A/B/C/D/E/F testing), you need to play in the sandbox given to you. Play with whatever toys you want, but don’t change the sandbox itself. It was built this way for a reason.
By the way, let me zoom out a little from this copy block. You’ll see this is the home page for IdeaPaint.com — a great company with awesome products and a kick-ass responsive home page. Now you can see the logo is in blue, too –the same blue as the headline copy, actually. And that orange button? It’s a complimentary color to the blue. See, designers do know what they’re doing. Don’t assume your five minute YouTube tutorial on Photoshop makes you a designer any more than watching Grey’s Anatomy makes me a surgeon.
Moreover, don’t try to be something you’re not. I wear a t-shirt every day of the week with a pair of jeans. I could wear a suit and tie to a meeting with a vendor, but that’s not authentic, nor is it a true representation of who I am. So I won’t entertain the notion of “testing” what response is better for my meetings — suit or t-shirt. Of course a suit is finer menswear, and while there may be a time and place for this, I’m ultimately a t-shirt guy. A better “test” is which t-shirt to wear: maybe the graphic on my tee or perhaps the age of the tee itself. My brand (the “Kyle Duford” brand) is t-shirt and jeans. Why change my essence?
Similarly, as you go test your site I encourage you to stay within yours. Stay true to the brand. Try to focus more on learning from consumer’s actions rather than how to impress them. That suit business will wear off soon enough — I’ll eventually get tired of matching ties to my suit or taking the whole lot to the dry cleaner — and I’ll soon enough revert to who I am: a t-shirt man.
Stay true to your business and test what’s needed. Remember, just because you can, doesn’t mean you should.