Secure search now obscures up to 60% of search visits, making it virtually impossible to determine which keywords are driving your site traffic. Appearances of the dreaded (not provided) have been steadily increasing. Now the default Google Analytics reporting is nearly worthless for understanding the impact and opportunities for your SEO program.
This has been driven primarily by the inclusion of secure search in Google Chrome and Firefox browsers for all logged-in Google account holders. If one of your website visitors comes from search while logged into Gmail, that referring keyword is hidden from your view. The same goes for a user logged into Google+, YouTube, Google Drive… You get the picture.
And while the percentage of hidden keywords varies from industry to industry, it has reached a point where large chunks of search traffic are distilled down into nothing more than a single line item in Google Analytics – an enigma wrapped in a riddle, as it were.
1. Making the Most of Your Marketing Efforts
If you’re investing time and money in search engine optimization, blogging, or other content marketing, secure search presents a big problem. How do you know what’s working and what’s not? You’re left back in the dark ages of marketing pioneer John Wanamaker, who famously said, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”
In a previous KISSmetrics blog post, Claire Broadley looked at three methods for gleaning insight into (not provided) traffic. In this post, we’ll look at four more ways you can crack the (not provided) code: dissecting Google search strings, surveying users, instrumenting your site search, and using query classification and advanced segmentation in Google Analytics. These techniques will give you additional tools to help you peer into the secure search black box and find the keywords that really matter to your business.
2. Dissecting Google Search Strings
One of the most challenging (but exciting!) ways to gain insight into (not provided) keyword traffic is to analyze the search strings of traffic coming from Google. If you look at the URL from a Google search in your address bar, you’ll see that it is not the actual URL of the page. Rather it is a redirect URL with a string of parameters and codes attached to the end. This provides Google a ton of information to help them refine and tweak their algorithms.
For example, when you search Qualaroo in Google, the click-through link in the result isn’t qualaroo.com; it’s actually:
This string provides a wealth of information about the keyword and relative link position of the result that was clicked. The two most interesting parts of the string are the “ved” and “cd” sections, where the codes correlate to critical keyword information.
Tim Resnick of Moz gave an incredibly detailed explanation of how to parse this information by configuring Google Analytics advanced segments. You can get step-by-step instructions on how to leverage this data in his post and webinar.
The “ved” portion of the URL provides information on the search vertical and the absolute and relative position of the keyword. You can use advanced segments in Google Analytics to parse out traffic based on these variables to get insights into the keywords and search results that are driving traffic to the site.
Resnick and the SEO community have divined some of the more popular combinations of “ved” variables in the table below. If you follow along in the original post’s comment thread, you’ll find plenty of links to additional research and discovery.
It’s easy to see the implications of parsing this referral data to help optimize your search efforts and content marketing. Say you’re driving a lot of traffic from images but not videos. This information will help you make decisions about how to enhance your multimedia content marketing strategy.
You can choose to combine this approach with the landing page strategy outlined in the previous post on (not provided) to determine which types of results are driving traffic to your top landing pages. Then you can set up goal funnels based on these parameters to see which optimization efforts are driving your real business goals: purchases, signups, and more.
While advanced, it does provide an inside look from Google itself on how and where your (not provided) traffic is coming from. But if that’s making your head hurt, there are easier ways to crack the (not provided) code.
3. Surveying Users
Using on-site surveys is a quick and easy way to ask search visitors how they found you. There are plenty of software options that allow you to quiz your site visitors. We humbly suggest Qualaroo, but the basic theory remains the same for all of them.
Configure your survey tool to display an onsite survey for visitors coming from only search engines. Some products, like Qualaroo, let you configure your settings to display the survey for users of only secure search. But even if it’s just targeted for search in general, you’ll get a more detailed picture of your most popular search terms as reported by your search-sourced visitors.
Here’s how this works. If you search for “product market fit” in Google, you’ll get my blog as one of the top results:
If you click through and your search is a secure one, you’ll get the following when you land on my site:
(Qualaroo for SEO showing up on Startup-Marketing.com for (not provided) visitors.)
Now I can see which keywords are recorded for secure search. I can use the relative weights of the responses to the number of nudge views to make some informed decisions about the keywords that make up the (not provided) bucket. For example, if 20% of survey respondents answer “product market fit” as their referring keyword string, it’s a pretty good bet that 20% of the (not provided) traffic is from searches for “product market fit,” and so on.
Best of all, you also can see which keywords and phrases drive conversions and purchases by configuring custom events and goals in Google Analytics to tie survey responses to conversions.
4. Instrumenting Site Search
Another easy-ish way to triangulate (not provided) keywords is to instrument your on-site search with Google Analytics. With this approach, you’re capturing the keywords used by visitors to search for information and products within your site to determine what keywords are important to the people that are using your site.
The theory is that people who use your site search are searching for things similar to those searched for by people using Google. While not perfect, it does provide a very clear picture of the types of information visitors think your site should have available. It’s another smart way to get inside the user’s head and find the keywords that matter to your visitors.
You can use this information to create content and optimize your site for the types of things people are looking for on your site. Using a similar methodology in analyzing survey results, you can determine the relative importance and prominence of keywords that your audience is searching for.
If one of your desired keyword phrases isn’t showing up in your site search, it probably means that either your audience doesn’t know the phrase is relevant to your business or the phrase isn’t very important to your audience. Both are actionable pieces of information.
You can instrument your site search in the Admin section of Google Analytics. The Google Knowledge base provides detailed instructions on configuring it properly.
Using Query Classification and Advanced Segments
This technique is another one for the more technically minded. It involves using Regex (regular expressions) and advanced segments to parse inbound (not provided) traffic into buckets that provide another level of detail about the traffic.
David Harry built on insights from Google’s resident analytics expert Avinash Kaushik to create this analysis. We refer you to the full post for the brain-spray awesomeness that is his analytics wizardry. The basic idea is to use broader query classification and advanced segments in Google Analytics to bucket users with similar intent, and then use the relative proportion of those visitors to determine the keywords that make up your (not provided) set.
Query classification is the practice of bucketing different types of keywords into similar types of visitor intent. For example, some users might be seeking specific types of information and use searches that help them discover that information; some users could be looking for specific brands and URLs; and others may be searching for specific items to purchase.
Query classifications account for the difference between searching for “eggs benedict recipes,” “epicurious.com,” and “where to find eggs benedict in Orange County.” David Harry’s table below gives you an idea of how this works:
By using query classifications, you can set up new advanced segments in Google Analytics which allow you to bucket users by search type. For example, you can create segments for your brand searches, head term searches, and long tail searches. This gives you the relative proportion of the types of search traffic you’re receiving.
Then you can look at the segmented search traffic to inbound landing pages to determine the proportion of search types hitting particular pages. Taking this relative weighting by page gives you an idea of the breakdown of (not provided) keyword traffic coming to those pages.
The power of setting up this kind of breakdown should be apparent. Now you can parse the (not provided) data set to understand, at the site and page level, which types of keywords are driving traffic and in which proportions.
Putting It All Together
The (not provided) keyword set will continue to grow in size as more and more users use Google products and more products integrate to secure search. But as we’ve just shown, there are creative ways, of varying difficulty, to crack the (not provided) keyword code and reclaim some of your data.
Using surveys, referral string analysis, and some smart segmentation, you can start to divine which keywords are driving your (not provided) traffic, allowing you to make smarter decisions about your search engine optimization and content marketing efforts.
In addition to the four ideas presented here, you can refer to the methods by Claire Broadley, and you can use the research by David Harry, Tim Resnick, and Avinash Kaushik as great jumping off points for further analysis opportunities.
Don’t be a modern-day John Wanamaker. It’s too expensive to not know where 50% or more of your search traffic is coming from. Get insights so you can get more out of your online marketing efforts. You’ll get better results, find more opportunities for improvement, and pass your competition, which is busy bemoaning the lack of insight into their search efforts.
Want some further reading on the subject? These posts are all super helpful in a continuing quest to crack the (not provided) code.
- 11 actionable posts on how to get not provided keyword data
- 7 fantastic seo tips for Google’s not provided keywords
- Dealing with not provided
Have another resource or idea? Share with us in the comments. It’s an ongoing process, and Google is constantly evolving. So stay on your toes and look for new opportunities to shed light on the (not provided) black box.
About the Author: Sean Ellis is currently the CEO of Qualaroo, a marketing software company that empowers marketers to better engage, understand and convert their website visitors. Prior to founding Qualaroo, he was the first marketer at Dropbox, Lookout, Xobni, LogMeIn (IPO), and Uproar (IPO) and also held interim marketing executive roles at Eventbrite, Socialcast, and Webs. Follow him on Twitter.