Kissmetrics Blog

A blog about analytics, marketing and testing

Built to optimize growth. Track, analyze and engage to get more customers.

5 Sources of Inaccuracy in AdWords Testing & Tracking (And how to eliminate them)

“Test, test test!”

It’s the unglamorous advice that no one wants to hear–except the few people who actually do it and seize the fistfuls of cash that lazier advertisers are leaving on the table. Let there be no doubt–sustained, serious testing is what separates the men from the boys in Google AdWords. It is what allows you to move beyond the theories and promises of e-books and instructional PDFs and see the reality of your actual, unique PPC campaigns. It is what guides you in making profitable tweaks and changes. But be careful! If you make any of these 5 mistakes, your test results could be inaccurate and lead you on a wild goose chase away from profitability!

  1. Statistically insignificant split tests
    Every AdWords marketer should be familiar with the concept of statistical significance. That is, if you are running 2 ads against each other in order to determine the higher-performing ad, you cannot make your decision after a day or two. Rather, you need to give each ad enough time to run so that when the results emerge, you can confidently say the ad with better stats is truly the better one. Without allowing this time to pass, how can you be sure the apparent “winning” ad is not just a fluke that benefitted from a random spike in clicks? If this sounds puzzling or time-consuming, don’t worry, it’s not.

    The free calculator at can determine in seconds whether you have enough campaign history to prove ad A really is outperforming ad B. Use this before making any split testing decisions on your Google ads!

  2. Not tracking profit per impression
    RocketClicks co-founder Glenn Livingston was quite possibly the first marketer to realize that profit per impression is the most important AdWords-related metric–not click-thru rate (CTR) or conversion rate, but which ad put the most money in your pocket each time it was displayed. As he notes in his PDF report on the subject [emphasis added]:

    “Google has everybody focused on CTR and conversion. But CTR + conversion individually don’t give you the right information to make your critical split testing decision. You might be surprised to note that Google isn’t really pricing their traffic according to CTR, even though that’s how you pay for it. All they have to sell you is space (impressions), so they’re doing the math behind the scenes to figure out how much money they make per impression.”

    It should go without saying that this is the number that counts. Fortunately, there’s a free web-based calculator that will let you determine profit per impression without putting together any spreadsheets or formulas. Use it!

  3. Prioritizing CTR above conversion rate
    An even deadlier testing sin than not tracking profit per impression is prioritizing CTR above conversion rate in your tests. This is a serious error, because as a PPC marketer you really need to be tracking dollars, not percentages. A high CTR in and of itself does not necessarily mean you are making money. In fact, it could mean you are losing money–after all, a high CTR only proves that lots of people are clicking your ad. If they are also buying your product, this is a good thing. But if they are only clicking your Google ad and leaving, your high CTR represents nothing but wasted money.

  4. Mixing content network traffic with Google traffic

    If you want to advertise on Google search results pages as well as the content network (the vast array of blogs and websites running targeted Google ads), you should maintain separate campaigns for this. The reason is that clicks from search engine queries and clicks from website ads are different sources of traffic. People are in a different frame of mind and a different place in the buying process depending on which of these ads they click and arrive at your website from. Therefore, grouping them both into one campaign and showing them identical ads serves only to distort your statistics and make it harder to determine which ads are truly making you the most money. Remedy this by going into your campaign settings and making sure only Google search is checked.

    (Note that this is not to say you should ignore the content network. Far from it! We are only advising that you maintain separate campaigns for PPC and content network so that you can accurately measure the results of each of these different traffic sources, rather than muddying the waters by combining them.)

  5. Letting Google “optimize” your ad rotation

    In all their benevolent wisdom, Google will, by default, monitor which of your ad(s) get the highest CTR and display those most often. At first glance this sounds helpful–who wouldn’t want to put their best foot forward and show their highest performing ad as much as possible? But this is actually not your goal. As an AdWords marketer doing serious testing, your goal is to continuously beat your winning ad, thereby pushing your CTR, conversion rates and profit per impression ever higher on an ongoing basis. The only way to achieve this is to give all your ads the same amount of exposure and let the best one win. Luckily, this is not at all difficult. Simply visit your campaign settings and check “Rotate: show ads more evenly” instead of “Optimize.”


About Kissmetrics

Kissmetrics combines behavioral analytics with email automation. Our software tracks actions of your users across multiple devices allowing you to analyze, segment and engage your customers with automatic, behavior-based emails in one place. We call it Customer Engagement Automation. Get, keep and grow more customers with Kissmetrics.



  1. Interesting article. I feel it is a good approach to let Google decide on Ad Serving.

  2. Can you elaborate on the significance of profit/impression? Where’s the big “ah hah?”

    I agree that putting CTR above conversions can be a mistake, but “can” is the keyword. Maybe you are only writing to people who manage LARGE AdWords accounts tied to LARGE businesses or purely ecommerce websites?

    In either of these scenarios, I “get it”.

    However, for many local businesses the number of conversions can be low. Businesses that take customer inquiries via phone often receive 6-9 phone “conversions” for every one AdWords conversion.

    I’ve seen business owners kill a particular keyword/ad with a high CTR and no (online) conversions — and lose a bunch of sales in the process because they weren’t taking into account the phone conversions.

    Of course… if you are relying on CTR over conversions then you MUST use some analytics program to continually monitor the quality of those visits. You are absolutely right – clicks that result in 100% bounce – are a complete waste of money.

    How do you reconcile conversions from returning visitors that might have come in the first visit via PPC, but return direct or organic using a branded search? I know that there is a way to track this, but are people using it??

    Great blog BTW – love all of your stuff.

  3. Ben,

    I think the “a-hah!” is actually contained in the article, specifically “which ad put the most money in your pocket each time it was displayed.” Too many advertisers decide the ad with the highest CTR or conversion rate is the winning ad, and delete the “loser”…but what they don’t know (unless they run profit per impression calculations) is whether that “loser” is actually making more money than the “winner.”

    Click-thru rates and conversion rates are just that – rates. But ultimately what you want to maximize is dollars – ie, profit. Profit per impression lets you decide split testing winners and losers on that basis.

  4. Point 2: surely it is profit per click that counts? Regardless of what calculations Google does behind the scenes, I pay an incremental cost per incremental click (not per incremental impression.)

    For example, I might have an ad with a very low CTR (so lots of impressions, but not many clicks), a very high CPC (so those clicks I do get cost me a lot per click), but a fabulous conversion rate (those few people that do click are the ones that really want my product). That could produce a very high profit per click (even though the clicks are expensive, I’m converting very many of them), but very low profit per impression (because there are so many impressions going without any clicks).

    Should I be dropping that ad in favour of one with higher CTR, lower CPC but lower conversion rate? Just because it has a higher profit per impression? Surely not? Surely profit per click is what counts?

  5. Haitham Al Humsi Mar 21, 2010 at 4:53 pm

    I need some help with one of my google adwords campaigns … and you seem to be a good person to ask.

    So I’ve been running a campaign for 3 weeks now and have as per adwords

    83 Clicks

    I am tracking my sales page / sales letter as a goal in my analytics and have imported those goals into adwords.

    Adwords is reporting 5 conversions to my sales page from 83 clicks

    how do these numbers ad up since the target page for my adwords ads IS my sales page, so theoretically i should have had 83 visits (not 5!!!) to that page corresponding to those 83 clicks…

    In my analytics those 83 clicks show up as 7 visits …

    So why am i paying for 83 clicks when realistically they have delivered between 5 and 7 visitors (depending on which report i trust ??) and where is the discrepency ?

    Very confusing …. :(

  6. Pablo Almeida Jul 26, 2010 at 8:40 am

    Nice tips! I already do this for some time and the results increases the campaign’s performance! ;)

  7. Hi there,

    Good article, though there are occasions when I would disagree with the first point, regarding statistical significance.

    Whilst you clearly shouldn’t make a snap decision based on a day or two of data, the most common significance level used seems to be 95%, which I believe can be too high here.

    If you set this level much lower (say 80%) then you will get results much quicker. Whilst this will mean that you will select the weaker advert sometimes (one in five times), you will be able to run far more tests in the same time period, so the incremental effect is likely to be greater (4 steps forward and one step back is better than 1 step forward).

    If an advert is a real dog, it will still come out behind at this significance level, so the only ‘wrong’ decisions that you make are likely to be the more similar adverts (in terms of performance).

    This approach isn’t right for every account – if you’ve got enough traffic to get significant results every few days, then there’s not much cost to waiting. But if your campaign has low traffic volumes, then waiting for weeks or months for statistical significance isn’t always the best long-term strategy…

    Regarding Bruce’s question regarding Profit Per Click, if you assume that conversion rates are the same in different positions, and that order values are also unaffected, then the less you pay per click, the higher your profit per click (since income per click is fixed). As a result, maximising your profit per click would result in reducing your bids until you got almost no clicks at all.


  8. The more I learn, the more I realise I don’t know!

    Conversion per Impression is a great idea metric to focus on.

    A slightly vague ad with a high CTR will likely be more expensive and less effective than a specific and relevant ad that has a lower CTR but higher conversion.


  9. Great..High CTR bring cost down and tracking and testing is everything I think. Thanks for the good tips.


Please use your real name and a corresponding social media profile when commenting. Otherwise, your comment may be deleted.

← Previous ArticleNext Article →