Your web analytics are wrong.

Well, not entirely. But, it is likely that you are spending a good portion of your research budget on collecting data while ignoring data that your firm already owns, collects, and is easily accessible.

Let me explain.

You’re a savvy marketer with the drive to get your product, your message, or your service in front of the right people. You’ve spent a great amount of effort creating brand recognition and driving traffic to your website.

You’ve probably hired new analysts and developers, paid for analytics services, and amassed a good amount of data. In this attempt to pull back the curtain of anonymity we call “the internet”, you’ve realized what many others have, as well…

Data is expensive. More accurately, turning data into useful information that provides actionable insights is expensive — and difficult.

Your enterprise must keep pace with a growing tech industry that waits for no one. This can be a challenge at times. In this fast-paced, demanding environment, data is generated at blinding speeds. Every web page, every app, every piece of software and hardware, the appendages of our organizations, and often times the front door, generates log data.

What is log file analysis?

Log data is simply data that gets stored in files by applications such as your web server. Each time a request is made to the server, details of the request are recorded.

For example, a client visits your website after finding you on a search engine. The process would look like this:

user-requests

Many times, this data is only used when a server goes down or there is a need to troubleshoot after a crash has taken place. However, it can be a bountiful source of useful information that helps you draw a more complete picture with your analytics.

Luckily, BI firms and analytics companies are changing the way this data is looked at and used. Through the use of data mining, machine learning, and other analytical techniques, the log files are becoming a superior resource for predicting information such as hardware failures, service demand, usage, and many other key metrics.

If you want to get deeper into it, I would recommend this great guide by Builtvisible.

What information is being collected?

Web server log files contain data about each and every file that is served, when, and to who. Best of all, your webserver already does this. You already have data about your web traffic sitting in compressed files on your server. This includes information that may not necessarily appear in your JS-based analytics.

With JS-based analytics, each web page needs to be tagged with the proper code to enable the collection of data. Tagging each page requires that each user allows this code to run in their web browser.

With all of the privacy concerns around web activity, many visitors use apps and browser extensions to block analytics code from executing, leaving you in the dark about their activity. This is not the case with log files. Each and every transaction is reliably recorded. This means that you can get a more accurate assessment of your website usage.

Since the information collected is from each request to the web server, as opposed to the user’s web browser as they view a web page, web logs contain different information. Some typical information included in log files are:

  • client IP address
  • request date and/or time
  • page or resource requested
  • HTTP code of the request
  • bytes served
  • user agent of the client
  • referrer of the linked request

These log files allow you to see non-human activity as well, like visits from search engine spiders. This can be incredibly good for SEO. Analysis of this data can help you identify holes in site architecture, identify weak (low-quality) or duplicate content, understand which content is served most often, and more.

Imagine this:

You are leaving the office exhausted from preparing the launch of a new flagship product going live on your site tonight. You and your team have worked countless long hours to meet milestones and deliver on time. You’ve stayed late, worked weekends, and even slept at the office a couple times. You tell your team to call you if anything major falls apart but really, you just want a good night’s sleep.

As you step through your front door after your hour-long commute home in horrible traffic, your phone rings. The stress washes over you as you see it is Jim from work. As you answer, you brace yourself for the bad news. But what you hear is completely unexpected.

Jim tells you that there was an error in the script controlling the checkout process for the new product, and the purchased inventory wasn’t being accounted for as it was bought. But it’s okay because the team caught this error in real-time, thanks to the new webserver log file analytics implemented last month. And, they were able to fix it immediately. It’s a good thing, too, because the new SEO rank of the site was driving in more traffic than ever.

This is just one of many practical situations that can play out from having better analytics on your web data.

Who is this good for?

Log file analytics can become a major source of insight for many firms. However, although ‘off-the-shelf’ software solutions are becoming more economical, analysts with the skills and knowledge needed to implement log file analysis are in high-demand, and staffing them comes with its own expenses.

Implementation can be cost prohibitive for smaller firms or firms that do not have a significant web presence, as their log files may not contain many useful insights. Depending on their webhost, some firms may not even have access to their web servers.

To best take advantage of the data locked away in log files, firms that have a significant web presence or conduct much, or most of, their business online, may find that this is a viable information resource.

For example, firms like Amazon.com would be giving up a major competitive advantage if they did not analyze log file information. On the other hand, your local drycleaners probably have no use for this type of analysis even though they may have a website.

Obviously, these examples are extreme cases, and your firm most likely falls in the middle somewhere. In any case, having a strong marketing analytics and visualization tool can help you gain the most actionable insights from this type of data.

Conclusion

While effective, JS-based web analytics solutions don’t provide a complete picture of your website usage. They are fallible and sometimes unreliable by themselves. By adding web server log file analysis, firms that have a significant web presence can get a more accurate assessment of how their site and services are used.

The cost of implementation for web log file analysis is dropping but still prohibitive to smaller firms, and they may not be able to realize a worthwhile ROI from its implementation. For firms that operate entirely online, web log file analysis could prove to be a cornerstone of their analytics infrastructure.