Slice & Dice – Dissecting and Understanding Log Files
Dissection is a useful tool in understanding any living thing. Pull it apart, peel back the skin and see what makes it tick. Since the Internet is as close to a living thing as an object can get, this axiom applies to it as well.Dissection is a useful tool in understanding any living thing. Pull it apart, peel back the skin and see what makes it tick. Since the Internet is as close to a living thing as an object can get, this axiom applies to it as well. Like an ecosystem, the Internet grows and mutates while parts of it die and rot, as do individual web sites.
To know if your web site is making you money, you need to dissect it as well. Who’s coming to it and from where? Do they buy or browse? Is the advertising you are paying for working? Are the search engines bringing you traffic? The answers to all of these questions and more are contained in your web server’s log files. Page requests, number of visitors and where visitors came from are contained therein, tagged by each user’s IP address.
However, log files mean nothing to most people, being massive records of undecipherable text. On lower end computers even attempting to open a log file can result in a system crash. Going through one by hand, in addition to being time consuming, reveals very little useful information. We’re talking about going through hundreds if not thousands of lines of code.
In order to make use of log files you will need an analytical tool of some kind that can decipher all of the contained information. The tools themselves can be complex and sometimes difficult to use, but their usefulness is apparent almost immediately. You’ll begin to get a grip on how many people are visiting your site and where they spend most of their time. If you make your money from advertising, this can give you a good idea of where you should place your sponsors’ links to maximize their benefits. If you make your money off providing content, you can quickly figure out what is most popular with your visitors and make it more accessible to them.
There are literally dozens of tools for this type of work, so I will go over just a few here. My personal favorite and an extremely easy to use analytical tool is the WebTrends Log Analyzer, v7.0, developed by NetIQ (http://www.netIQ.com). The interface on this makes it accessible and the tutorial is actually useful. The information that it retrieves is extremely customizable and the reports it generates are easy to read. Even if you didn’t know what a log file was before reading this article, you can use NetIQ’s product. Perhaps best of all, they have an extremely capable and friendly support staff that actually answers the telephone. WebTrends Log Analyzer is also expandable, capable of deciphering the log files from multiple servers. If you plan on expanding your operation any time in the near future, you won’t have to worry about upgrading to another product. The down side to NetIQ’s product is the price tag; $499. Also, the ‘Essential Care Support Plan’ is an additional $89, and I highly recommend getting it.
In the event that this is out of your budget, you may want to take a look at 123LogAnalyzer (http://www.123loganalyzer.com). 123LogAnalyzer is only $129.95, but can only support information from one server. The price and ability make it perfect for any small operation. I cannot say that I’ve ever used it myself in a professional capacity, but I have several colleagues that swear by it. It is extremely simple to set up and has an easy to understand HTML interface. It is, however, considerably more limited in its reporting capacity than WebTrends LogAnalyzer. To give you an idea, in WebTrends you can personalize up to 50 different types of reports. In the case of 123LogAnalyzer the reports will give you all the general information that you need, but nothing to that level of detail.
If you don’t want to plunk down cash to purchase large pieces of software like the two above, there is an alternative. There are several firms out there that use ASP technology to accomplish similar goals. For a low monthly fee, you can gather nearly as much information from ASP services as you do from software log analyzers.
ASP services, such as HitBox Professional (http://hitboxprofessional.com) and SuperStats (http://www.superstats.com), collect this information by having the client place a few pieces of code on each page that he is interested in monitoring. The code is very small, so it doesn’t affect loading time, but it does depend on JavaScript and Cookies being enabled by the surfer. While this may limit some information, it is cheaper. HitBox starts at $24.95 a month, SuperStats starts at $19.95 a month.
This is just the tip of the iceberg. There are many different log analyzers and ASP services, all of them with different advantages and disadvantages. Regardless of what those may be, though, any webmaster that plans on turning a profit needs the information provided by these tools. So do yourself a favor, grab your scalpel and get to work.
Matthew McLean has been the director of marketing for Webroot Software since 1998. (http://www.webroot.com). Webroot Software publishes security, privacy, and Internet enhancement software designed for the home or office user. He is currently working on expanding Webroot’s affiliate program. To learn more, please write mmclean@webroot.com.