How much do you trust your website stats, e.g. Google Analytics?
The answer should be ‘not that much’.
Data from three separate UK membership organizations reveals a steady decline in the proportion of visitors who are captured by Google Analytics.
Initial data discrepancies
Year Out Group were concerned their website visitor numbers in Google Analytics have been dropping year-on-year since 2013. This is despite strong search engine rankings and a new website launched in March 2018.
The new Year Out Group website is hosted withWP Engine. WP Engine provide detailed analytics on the site traffic based on their log files. These log file stats are processed to filter out non-human requests(e.g. search engine bots) from real visitors – just like Google Analytics tries to do.
I decided to compare the traffic reported by WP Engine’s log files with that in Google Analytics.
Web traffic analysis is complex. Each available data set will include some form of filtering or processing. I fully expected some variation between the two data sets in absolute terms, but thought the general trends would be broadly consistent.
I was wrong.
And this is where it gets interesting.
The WP Engine logs show a strong trend of visit numbers increasing, while Google analytics shows a continued gradual decline:
Year Out Group stats. Google Analytics shows a decline while WP Engine logs show a growing audience.
Some variation in the actual numbers is one thing.
But when the two sets of data are reporting fundamentally different trends, there are huge implications.
Further proof Google Analytics is inaccurate
The next step was to see if other clients with the same WP Engine logs available showed similar characteristics.
All three datasets show large differences between visitor numbers. Google Analytics is only capturing 20-80% of the visit numbers from the log analyses.
More crucially, the proportion of log file visits reported in Google Analytics is decreasing for all three sites. The sites with the highest percentage of log file visits in Google Analytics show the sharpest declines:
Proportion of log file visits recorded as Google Analytics sessions.
Note the downwards trend across three separate websites.
(The data points with ratios over 100% are due to Cloudflare caching. See notes for details).
Over the same time period, all three sites show strong growth in visits from the log file data while their corresponding Google Analytics data report visits as either static or in decline.
We have to trust that data available from Google and WP Engine are accurate. There will always be an element of filtering and processing of these data.
I am working on the assumption that all such processing is applied consistently.
Ad blockers, privacy settings and network filters
Google Analytics relies on the user’s browser to send data to it:
How Google Analytics works under normal circumstances.
Clearly if this information is not sent, their usage will not be tracked in Google Analytics.
Many ad blockers – whose purpose is to hide adverts on the websites you visit – also block tracking and analytics services, including Google Analytics.
Like what you've read? There's plenty more. Subscribe today.
These articles from our thought leaders are designed to be seeds of discussion. Our community consists of association professionals who are passionate about our industry, and are coming together to share ideas and opinions. You have valuable insights from your own experience, and we enthusiastically invite you to join our community!