Kissmetrics Blog

A blog about analytics, marketing and testing

Built to optimize growth. Track, analyze and optimize your growth cycle.

Which Matters More: Traffic or Testing?

In recent years, there has been a shift in attention from traffic to testing.

It makes sense, it’s becoming increasingly hard to win at online marketing. In many ways, it seems like the easiest way to get more out of your traffic is by improving site performance.

As exciting as the idea of conversion rate optimization is, though, the question remains:

Which affects your marketing campaign success more—traffic quality or site quality?

It’s an important question to consider. For most marketers, time and resources are limited, which means you need to spend the majority of your time in the areas that produce the best results.

So, if you have to choose between refining your traffic and perfecting your site, which optimization will give you the best bang for your buck?

Running the Numbers

To evaluate how testing and traffic affect your marketing success, let’s run a quick hypothetical.

Traffic

In this scenario, let’s suppose that you are running paid search ads for your business. You’ve got a decent site/landing pages and—on average—you pay $4 per click.

Recently, you spent $20,000 on a new campaign that produced 5,000 clicks.

5k-visitors

That’s a lot of traffic, right?

Unfortunately, there’s a problem. Most of that traffic isn’t actually interested in what you’re selling.

As it turns out, the average paid search account wastes 76% of it’s budget on search terms that never convert. In other words, 76% of paid search ad spend goes towards the wrong traffic.

For you, that means you spent about $15,000 on clicks that have no chance of converting.

Ouch.

They may have accidentally clicked on your ad…or they thought you were selling something different than you actually sell. Maybe you were simply bidding on the wrong keywords.

Thanks to all those irrelevant clicks, only 1,250 of your 5,000 site visitors are actually potential customers.

So, in terms of your relevant traffic, here’s what your ad spend really paid for:

actual-visitors

Hm, that’s not nearly as exciting. You might be paying $4 per click, but you’re actually paying $16 per relevant click.

But still, that’s nothing a little testing can’t take care of, right?

Testing

Now, as a best practice sort of marketer, you ran an A/B test on all of that traffic.

Unfortunately, what you didn’t realize was that you weren’t testing 5,000 visitors. Remember, only 25% of your traffic is actually interested enough to potentially convert.

As a result, you thought you were testing this:

5k-test

When in reality, you were testing this:

relevant-test

That’s unfortunate, but you’ve still got enough traffic to run an effective test.

During your test, you get 100 conversions in your control (variant A in your A/B test) and 120 conversions from your variant (variant B).

relevant-test-results

Sweet! Clearly, your interested traffic responded much better (20% better, to be precise) to your variant.

At $5 per click, you just dropped your cost-per-conversion from $25 to $20.83. The results are statistically significant, so that’s a win, right?

But wait, before you start throwing confetti, remember, you didn’t pay for 1,250 relevant clicks.

You paid for 5,000 clicks.

Here’s what your test population actually looks like:

actual-traffic

All of a sudden, your results don’t seem nearly as exciting:

actual-test-results-1

Your test is still a success. Your conversion rate went up by 20%. However, all the “wrong” traffic you paid for has diluted your conversion rates to 4% and 4.8%.

And here’s the real problem—your shiny, new, optimized cost-per-conversion is $83.33.

$83.33?

But wait a minute, when we were only looking at relevant traffic, the cost-per-conversion for your control was only $25!

That means your “optimized” cost-per-conversion is over 3x what you would have paid if you were only paying for the right traffic—even without testing.

Sure, your test helped reduce your cost-per-conversion, but you can’t fix your traffic by testing your website.

What Happens When You Stop Paying for the Wrong Traffic?

So, if you can’t test your way out of the wrong traffic, what if you stopped paying for those irrelevant clicks?

Well, over the years, we’ve used this tactic countless times for clients.

Here’s what happens:

cost-per-conversion-vs-wasted-ad-spend-570-px

 

As you can see, as you waste less money on the wrong traffic, your cost-per-conversion drops…exponentially.

For this particular client, reducing their wasted ad spend from 91% to 68% cut their cost-per-conversion from $160 to $39.

And it happened in a matter of weeks.

Now, don’t get me wrong. I’m a big believer in the power of testing. However, every successful test starts with the right traffic.

And the Winner Is?

So, traffic or testing…where should you be focusing your efforts?

To be honest, it really depends on where your campaigns are at. If your online marketing is putting the right traffic on your site, a great testing strategy can help you squeeze every last conversion out of your clicks.

However, if you’re like most business, you’re probably wasting most of your budget on the wrong traffic. In this case, dialing in your marketing campaigns will deliver the biggest and fastest results.

Here are 3 things to start with:

  1. Define your audience. Who is the “right” audience? What are their pain points? Do your ads speak to the right pain points?
  2. Review your targeting. Does your targeting match your audience profile? Are you targeting the right demographics?
  3. Use your analytics. Are your clicks producing conversions and sales? Which marketing channels are producing the best results? Do your results justify your spend?

Once your marketing is primarily driving the right traffic to your site, you can then use testing to really optimize your campaign performance.

And, since you’ll be testing the right traffic, your results will be truly meaningful to your business.

Conclusion

Obviously, in a perfect world, you would have the time and resources to work on both traffic and testing. In the real world, though, sometimes you have to prioritize.

Having used both testing and traffic optimization for years to improve marketing performance, I’m definitely a big fan of both. However, testing works best when you are optimizing for the right traffic.

So, if you can only pick one, it makes the most sense to tighten up your traffic…and then start testing.

Traffic or testing—which would you pick? Where do you spend your time and why?

About the Author: Jacob Baadsgaard is the CEO and fearless leader of Disruptive Advertising, an online marketing agency dedicated to using PPC advertising and website optimization to drive sales. His face is as big as his heart and he loves to help businesses achieve their online potential. Connect with him on LinkedIn or Twitter.

  1. What I appreciate most about this article is that it’s trying to combat common assumptions about site traffic.

  2. This is a great display of why it’s not just the numbers that are important, but looking at the right numbers. Not taking all the factors into account can lead to wildly varying conclusions.

  3. Great stuff Jacob. Had initially browsed through but then came back later for a thorough read. This was a great article. Never heard of it from this perspective before, and with data for comparison.

  4. Great post. 100% agree. The only issue is that people do sequential testing and are not aware of the factors that may skew the results, such as seasonality which you mentioned.

    This post is a great reminder. Thanks!

  5. Alberto Dominguez Jul 18, 2016 at 8:39 am

    Thanks for the post. It reflects the “perspective shift” that is necessary to undergo from time to time in order to evolve our campaigns. We are so often so caught up in the optimization process that we usually oversee that a core assumption is just not working.

    However, in my experience with underoptimized accounts, there’s a lot more of a back and forth process. Sometimes, you can be perfectly sending the right traffic to your site and still see low performance for some keyword variations (due to poor product information, competitive pricing, or whatever any other issue).

    If that’s the case, cutting the traffic to soon reduces the amount of data generated and it makes it even more difficult to assess where the problem is.

8 comments

Please use your real name and a corresponding social media profile when commenting. Otherwise, your comment may be deleted.

*

← Previous ArticleNext Article →