Stop Google Analytics From Stealing Your Valuable AdWords Keyword Data

Are you a Google AdWords advertiser using Google Analytics? STOP! You MUST read this post because you are losing money daily and we are going to help you stop the bleeding.

There is a problem with the default functionality of Google Analytics when used in conjunction with AdWords. Google Analytics (GA) doesn't report the actual phrase a shopper entered into the search bar, only the keyword phrase you are bidding on.

Let me explain:
- You bid on the keyword 'shoes' using 'broad match'
- A shopper searches for 'blue suede shoes'
- The Traffic Sources > Keyword report in GA shows the search as just 'shoes'

Even worse, Google likes to use synonyms when your terms are under the broad match type (called automatic matching or extended broad match).

- You are bidding on the keyword 'running shoes' using 'broad match'
- A shopper searches for 'Adidas Gazelle'
- Google shows your ad, but wait, you don't carry Adidas shoes

Why would Google do that?

The shopper searched on blue suede shoes, not shoes! You don't sell blue suede shoes. You have been making decisions based on inaccurate data.

Follow these simple steps to start seeing the EXACT phrases people are using when they click your AdWords ads.

It will help you find terms to add to your negative keyword list. You can also start honing your ad and landing page copy to better reflect how shoppers search.

The Google Analytics Exact Query Solution...

This solution comes from our friends at VKI Studios, a Google Analytics Authorized Consultant and overall great bunch of people (see their analytics blog for some great tools and tips). Specifically, Brian Katz. They have evaluated various means of cracking this nut, and we have their final solution. Credit and comparison of other methods are at the bottom of this tutorial.

1. Create a new Google Analytics Profile

We do NOT want to overwrite any core data, so a new profile keeps everything intact. Even Google says it is a good idea.

Select Add a Profile for an existing domain, select which domain, and enter any name for the profile you choose - the more descriptive the better. You will not have to add any tracking code or tag anything, so no need to get the ponytail guys involved.

2. Create the first filter

Locate your newly created profile and click Edit under the Settings column. Then click Add Filter.

Field A -> Extract A: Referral: (\?|&)(q|p|query)=([^&]*)
Field B -> Extract B: Campaign Medium: (cpc|ppc)
Output To -> Constructor: Custom Field 1: $A3

3. Create the second filter

Locate your new profile again and click Edit under the Settings column. Then click Add Filter.

Field A -> Extract A: Custom Field 1: (.*)
Field B -> Extract B: Campaign Term: (.*)
Output To -> Constructor: Campaign Term: $B1 ($A1)

As with almost all multi-part filters, sequence is critical and must be ordered accordingly using the "Assign Filter Order" page for the profile.

That's It!

Here are what the results should look like when you run the Traffic Sources > Keywords > Paid report in Google Analytics:

The following set of results were obtained using an in-line filter to show bid-terms that would be different from the search terms

An unfiltered result would look as follows:

The above technique provides useful data as is but it does have some shortcomings in that it does not associate the newly overwritten Campaign Term field with Transactions, as is shown in the following screen shot:

It is probably the result of using session-based values (e.g.: all the Campaign fields) and pageView-based values (e.g.: Referral). Caught in the middle are the event-based eCommerce transactions.

In his book "Advanced Web Metrics with GA" (Page 199) Brian Clifton documents a method attributed to Shawn Purtell of ROI Revolution that uses 3 filters to show each Transaction with its bid and search terms appended.

Hat Tips to Others Tackling this Problem

The original solution for this came from Brian Clifton, formerly of Google.

The solutions (Using Filters):
- How to Get Detailed PPC Keyword Data from Google Analytics
- NUDE: AdWords Keyword Data Exposed With Google Analytics!

An updated solution from ROI Revolution (Using JavaScript):
This solution uses the User Defined variable so it won't be appropriate if you're using the User Defined variable (created with _setVar()) already
- Exact Keyword Tracking with Google Analytics, Revisited
- Exact Keyword Tracking with ga.js

Comparison of the two methods

I checked out the two methods (Filters vs. JavaScript) . Since readers commented saying the filters did not work or "no longer worked", I took a closer look. The devil is in the detail. Errors in their implementations may have been the cause of the malfunctions.

JavaScript vs. Filters

JavaScript
The two methods both extract data from the and Referrer and Campaign Medium checking the latter for "ppc" of "cpc" using regular expressions. They both concatenate the bid and search terms. The JavaScript method goes 1 step further by looking for the gclid value unique to Google AdWords. That may also be done in the filters but I don't believe it would enhance the filter solution.

The JavaScript performs its magic at run time. It uses the "troublesome" _setVar() cookie to store the bid and search terms in the User Defined field. It does so using a generally accepted "kludge" to work around _setVar()'s issues (a topic all of its own).

The greatest disadvantage to this method is that it monopolizes the User Defined Value. With all its troubles, it is an invaluable resource that most will (should ?) be using to segment visitors. Since it is stored in a domain specific cookie it cannot store profile-specific data to different profiles (well, it can be pushed to greater limits but that is a blog post all of its own).

It should be possible to rewrite the URL of the landing page before ga.js writes the Campaign cookie (again a topic of its own)

Filters
The filters run at data processing time and so, I expect those may prove marginally more reliable than JavaScript and cookies (although all subsequent visits from the AdWords campaign will rely on the keyword and other campaign data being extracted from cookies by ga.js or urchin.js) so that is no reason to choose one above the other.

By default, however, I am biased in favor of filter-based solutions because they are independent of the implementation and so don't require updates to a site's GA coding. Implementation is quicker and easier, as is propagation of the solution across profiles and GA accounts. In fact, in the time it takes to update the code on some sites (those that are not tagged as efficiently as they might have been) or in the time to get a site's 3rd party developers to make the changes, a GA consultant could implement the solution for a number of accounts, regardless of the level of access the consultant has to the coding.

Note: Analysis and much of the technical write-up done by VKI Studios, Brian Katz