Kissmetrics Blog

A blog about analytics, marketing and testing

Built to optimize growth. Track, analyze and engage to get more customers.

8 A/B Split Tests That Made Shocking Discoveries

If your goal is to squeeze the maximum amount of profit from your business, then you want to boost your conversions. A great way to boost conversions is to split test and find out what works best.

However, split testing isn’t always black and white.

Every business is unique, and sometimes common marketing wisdom can fail. Here are 8 A/B split tests that had either shocking results from simple changes or results that defy common marketing knowledge:

1. 400% Conversion Boost by Removing Security Badge

Conventional marketing wisdom tells us that adding images of security badges and safety certificates increases conversions by creating a sense of security. This case study proves otherwise.

Web designer Bradley Spencer wanted to test the power of “security” assuring images and badges in the sidebar of a coupon site he was working on. He wanted to see if there was a link between the security badge and the number of people who clicked on the coupon button.

So, he removed the security badge and experienced some surprising results.

This is the site before removing the security badge:

point1b

This is the site after removing the security badge:

point1

The results?

In just a few days, he experienced a whopping 400% increase in conversions. Four times more people clicked on the coupon link.

Are you using any safety assuring badges on your site? If so, split test a page without the assuring images and see what happens. You might end up with surprising results yourself.

Source: http://www.bradleyspencer.com/2010/5-surprising-lessons-from-user-testing/

2. 115% Conversion Boost with Complementary Landing Page Copy

Strong copy is king. It will always beat average copy, right? Well, not really. A case study by California Closets shows otherwise.

Take a look at these two pages:

point2

Version B seems like it would easily outperform A, doesn’t it? The headline is snappier and the CTA seems stronger and more urgent.

But, test results revealed that version A, with its weaker copy, boosted leads by a chunky 115%. How?

Despite having weaker copy, version A was designed to complement and relate to the PPC ads that drove users to the page.

The lesson here?

While strong copy is a powerful marketing tool, never underestimate the value of having a sense of relevancy in your sales funnel.

Think about where you traffic is coming from. Is your landing page consistent with what your prospects view before arriving?

Source: http://whichtestwon.com/archives/17969

3. Over 38% Rise in Conversions from Tweaking Button Copy

When trying to optimize conversions, it’s easy to pay attention to the flashy details that everyone talks about – headlines, subheads, images, and copy.

But, what about the small details that often get overlooked?

For instance, what about button copy?

While it seems like a small thing to focus on when compared with other potential changes, tweaking your button copy can have some awesome results.

point3

When Michael Aagaard wanted to increase conversions for a client, he tested an interesting hypothesis. He determined that the more value you can convey, the higher your conversions will be, even if it’s the value of something simple like a button.

So, he changed the button copy of the control from “order information” to “get information,” thereby focusing more on what the customer receives instead of what the customer has to do.

This simple change resulted in a 38.26% rise in conversions.

Aagaard also claims to have achieved conversion lifts of between 5-200% by “simply tweaking words.”

The above case study shows how important conveying benefits to your customers is, even if it’s through your button copy.

To optimize your button copy, think of what your customers want out of your offer. What exactly are they looking for when they click that button? Then, proceed from there.

Are you still looking for more ways to strengthen your CTA copy? I’ve got you covered.

Source: http://contentverve.com/how-changing-1-word-in-the-call-to-action-generated-a-38-26-lift-in-conversions/

4. Over 80% Increase in Free Account Signups by Simply Reducing Distractions

point4

It’s easy to fall into the trap of thinking that a smooth, good-looking design will boost business. But, that isn’t always true. Which of the landing pages above do you think converted better?

If you guessed version B, you’re right.

Even though version A seems more open and creative with its mini speech bubbles, it was no match for the straight-to-the-point forwardness of B. The three solid bullet points make the benefits easy to read, hook the viewer’s visual attention, and quickly convey value.

The lesson here?

Try not to be too artsy with your design, keep it simple. Make things as visually easy as possible, and focus on your prospect.

Source: https://whichtestwon.com/test/which-page-got-89-8-lift-in-free-account-signups/

5. Over 57% Increase in Leads by Introducing a Search Box Drop Down Menu

This test was performed on the Casa Mineira real estate company website. It doesn’t rebel against common marketing wisdom, but it created a powerful increase in conversions from a simple change.

point5

The conversion goal being tracked was the number of email signups from each variation.

By simply adding a drop down menu, the company received 57.25% more email leads.

The variation won because it gave visitors a simple path of action to follow.

In the standard search box, visitors had to be sure of what they were looking for, and they had unlimited options of keywords to search.

This shows that too many options can harm your conversions.

For another example, take this study by The New York Times. It was designed to test the influence of choices and options on buying decisions.

On a day when 24 different flavors of jam were available in a California gourmet market, 60% of people tasted the jams. Only 3% actually bought anything.

On a day when 6 flavors were available, however, while only 40% of people stopped to taste the jams, 30% of them made purchases, resulting in 600% more jam sold.

More doesn’t seem better now, does it?

Source: https://vwo.com/blog/split-testing-standard-search-box-drop-down-search-increased-leads/

6. 304% Boost in Conversions by Moving the CTA

Recently, studies have shown that content placed above the fold attracts 80% of a consumer’s attention. So, it makes sense for marketers to use this as a reason to place their value propositions and CTAs above the fold.

(If you’re not sure what “above the fold” means, it’s the part of a website or landing page that is visible without having to scroll down.)

This best practice doesn’t always mean optimum results.

Check out the images below:

This is the result of a test in which Michael Aagaard cranked up conversions a staggering 304% by simply moving the CTA below the fold. Aagaard says this is because of a direct correlation between the complexity of a product and the location of the CTA on a landing page.

It’s similar to the copywriting rule of length, don’t you think?

The more expensive and more complex your product, the longer your copy will be.

Similarly, the more expensive and complex your product, the lower your CTA should be.

As always, test to find out what’s best for your business.

Source: http://contentverve.com/how-moving-the-call-to-action-below-the-fold-generated-a-304-lift/

7. 100% Boost in Leads by Showing Price on Landing Page

SafeSoft Solutions is an ecommerce site that develops products for customer contact centers. They wanted to test the effect that showing price on the landing page would have on conversions.

So, they ran a split test, one page with the price boldly displayed, and one without.

point7

Now, most people would guess that the page with no indication of price would beat the variation. But, that’s what makes this test interesting.

The page with the price prominently displayed caused a 100% increase in lead generation.

The reason?

Sometimes people think that if a company doesn’t show pricing, the product they’re selling must be expensive. Visitors fear that means they’ll have to talk to a salesman, or worse, endure a sales pitch to get information.

This test reveals that showing price can actually work in your favor. It removes doubts and questions and clearly displays what the prospect can expect to pay.

Source: https://vwo.com/blog/prominent-price-display-increases-leads-generated-by-100/

8. 102% Rise in Opt-ins from Removing Social Proof

A popular marketing assumption is that people look for the approval of others before biting the bullet on a buying a decision. This gave rise to providing social proof and popularized the routine as a best practice.

However, when Derek Halpern split tested his sidebar opt-in form for diythemes.com, the results were surprising. Contrary to best practices, social proof hindered conversions.

point8

Halpern found that by removing social proof, sidebar opt-in form conversions increased by 102.2%.

Why did social proof hurt conversion rates?

According to Halpern, it either served as a distraction and complicated the process or the social proof numbers weren’t big enough. The first theory makes more sense. When you look at the opt-in forms, the two with social proof simply seem like they demand more energy to interact with, don’t you think?

Again, this is another reason for you to split test any methods you’ve based on best practices.

Source: http://diythemes.com/thesis/increase-conversions-split-testing/

Conclusion

The above split tests have pretty interesting outcomes. They show that just because something is a best practice doesn’t mean it’s best for you. So, never stop split testing to find out what works best for your business.

Have you run any split tests that delivered odd results in your business? Tell us about them in the comments below.

About the Author: Hassan Ud-deen is a freelance blogger and copywriter who helps businesses use content to grow. You can find out more about him on his blog www.f-bombmarketing.com. Or if you need help with your blog posts or copy, shoot him an email or connect with him on Facebook and Twitter (@Hassassinz.creed).

  1. The conclusion about the security badge is I think not supported by the tests — it isn’t the badge, it’s the gigantic green attention-stealing thing. Removing the brightest most visible element on the top of the page that drew the eye first meant people looked at the next-most visible thing instead: the coupon. Trust certificates have their place, perhaps, but maybe just not so prominent?

    • The first thing that came to my mind is that most sites that feature security badges and lots of green buttons and safety indicators are websites that carry unsafe content like torrents, cracks and file-sharing websites.

    • Yep – that’s what I’m thinking. But the real lesson IMO is to go down a road of removal. How far can you take removing “stuff” and how much does it help?

      • Hassan Ud-deen Apr 12, 2015 at 11:07 pm

        I’d be very interested in seeing a few tests that focus on removing “stuff”. I wonder what the results would be

    • Hassan Ud-deen Apr 12, 2015 at 10:57 pm

      Hey Zoe

      Yeah, I get where you are coming from. It is likely that it either steals more attention than necessary, or reminds people of scam sites where they might have been “taken”.

      It’s all about your prospects at the end of the day.

  2. I think everything need to be A/B tested.
    Even each plugin for wordpress visible for visitors need to have A/B testing feature.
    Btw. I was surprised about #7. I thought people more curious about the price will click next button.

    • Hassan Ud-deen Apr 12, 2015 at 11:12 pm

      I agree Yaro, everything should be split tested for maximum impact. But, I think the something to keep in mind is; what to split test and when?

      Starting with bigger, more influential things will yield a more ROI.

    • I have to agree with you Yaro, I suspected the same thing fro #7. Goes to show the power of A/B testing even the things that seem obviously correct.

  3. Christian Haugaard Brix Apr 09, 2015 at 10:29 pm

    Nice examples. Very bad images though – hard to “judge” the details based on them..

  4. Hassan, this post is excellent – and a real eye-opener. Thank you.

    One comment, if I may – point 6 appears to refer to images, but they seem to be missing..?

    • Hassan Ud-deen Apr 12, 2015 at 11:15 pm

      Hey Mark,

      Thanks for reading. I hope you enjoyed reading it as much as I enjoyed writing it.

      I’m not too sure about the image, probably a little hiccup.

  5. Jennifer Lamb Apr 11, 2015 at 9:25 am

    thats great

  6. What’s the best way to do a split test? Are there good plugins or best practices for setting up the tests?

    thanks

13 comments

Please use your real name and a corresponding social media profile when commenting. Otherwise, your comment may be deleted.

← Previous ArticleNext Article →