Not Provided FrustrationHaving enjoyed a long weekend in the majestic serenity of Cornwall, I have returned to a maelstrom of online angst aimed at Google.

Although not yet officially announced, it looks as though Google has now turned on encryption for all users, whether they are logged in or not.

Whilst I have written about my frustration over the dreaded ‘(not provided)’ phenomenon in the past, I can’t really say that I am that surprised by this move and there is a sense of inevitability about it all.

There are various schools of thought doing the rounds in terms of the real motives for this move, from the US National Security Agency furore to the accusation that it is nothing more than a (very) thinly veiled attempt to increase advertising revenues, but this is an issue for all in the SEO world that is not going to go away.

The rise of (not provided)

Since Google originally announced the move to encrypting searches (initially only for logged in users), we have seen a steady increase in the percentage of searches that have ‘(not provided)’ as the keyword.

The following graph, which shows the search volume for ‘(not provided)’, is typical for most Google Analytics accounts that I see:

(not provided) search volume

It has been growing steadily and a lot of accounts are around 75% already, so the latest news is not actually as dramatic as many are shouting about. It is not just Google that has been making things difficult as Firefox also followed suit and configured https searches by default.

Unfortunately, I think we just have to accept that it will now be 100% and the keyword information in any web analytics package is now largely useless.

What can we do about it?

The core frustration is that it is becoming very difficult to really measure the success of any SEO initiatives that target specific keywords. Whilst the days of targeting a handful of keywords are long gone, it is important to monitor the visibility of keywords to justify the investment made in your content and outreach.

Ironically, I think that this is actually going to push people back towards ranking reports. For me, this is a great shame as I have always preached that analtyics is a much better measurement device than ranking reports, which are far too susceptible to personalised fluctuations. How can we now encourage clients to use analytics rather than ranking reports, which may well be generated by scraping data (Google doesn’t like this…).

There have been workarounds for estimating the volume of brand searches (e.g. this post by Econsultancy) but these are now fairly redundant if ALL searches are hidden.

There are, however, two reports that you can use to try to help make sense of the madness. Both are provided by Google themselves.

1) Google Webmaster Tools (GWT)

For some time now, Google’s Webmaster Tools has included a report that sheds light on which keywords are performing for your site, as shown in the example below (actual keywords not shown for confidentiality reasons):

GWT report

This report, called ‘Search Queries’ is accessed via the ‘Search Traffic’ tab. You can also select a more advanced view which will show you changes in the data, which serves as a useful barometer to measure trends.

2) Google Adwords

Google announced a new ‘paid & organic report’ recently that allows you to “analyse and optimize your search footprint”.

Whilst you will need to have an Adwords account to use this report, there is no need to actually have any active campaigns running, so it is essentially a free tool.

It is not the easiest report to find, but sits under dimensions > View : Paid & organic, as shown below:

organic paid analysis

Whilst it is not as easy as just looking at keyword data within your chosen analytics package, this is an interesting report and also allows you to measure the impact of having both paid and organic search visibility.

What next?

To be honest, there isn’t much we can do about this. The SEO community has been moaning about the lack of transparency over keyword data since 2011 and it had no effect whatsoever. Google has never made it easy for us poor SEO folks but it is important to embrace the data that IS made available rather than cry over the good old days. It may be more hassle to go looking for it, but you can’t deny that Google Webmaster Tools  is becoming a lot more useful and I do feel that Google has a commitment to improving it.

I do, however, worry about the accuracy of some of the data in GWT. Whilst it is true that an average rank is actually a lot more useful than data from most rank checking platforms, I can’t help but feel that it is very wrong sometimes.

There is also no doubt that it may well push more organisations towards Paid Search, where the concerns surrounding privacy miraculously evaporate, as you can see exactly which keywords are working well for you. It is hard not to be cynical about that, but you shouldn’t feel scared of PPC as it can be managed very carefully (as long as you know what you are doing – you know where to come if not!).

What I really want to see is an update from Matt Cutts, who originally told us that the encrypted search issue would affect less than 10% of searches. Errr, I think you need to add a zero to that?

What do you think? Blatant commercial exploitation from Google? An inevitable response after the privacy concerns? A storm in a tea cup? I would love to hear your views so welcome your thoughts below.