Home » Data Protection » Network Security

Another Look at Log Files

These long-standing logs can help you monitor your networks and employees. So before you invest in a new kind of data collection system, review your log files.

April 01, 2005 — CSO — Marcus Ranum architected the first commercial firewall in 1990. He founded Network Flight Recorder Security, the company responsible for the first network forensics tool. And last summer at the Usenix conference, during a course he was teaching on log file analysis, he said that if nobody is ever going to look at your log files, then you might as well not bother keeping any logs at all.

For those who don't know him, Ranum is one of those security professionals who loves to be inflammatory. He maintains a webpage advertising an "ultimately secure intrusion prevention system"; it's a wire cutter. His most recent book, The Myth of Homeland Security, lambastes the government's "knee-jerk security at any price" response to 9/11. Ranum says that what we have bought for "nosebleed prices" are nothing more than "feel-good" security measures.

So when Ranum says that most log files are useless, and that most organizations are better off deleting their files and saving the disk space for something that's actually productive, he isn't really arguing for widespread file erasure. Instead, he's rightly sounding the alarm that organizations need to do a better job of working with the security tools they already have, such as log files.

This is especially true today because log files are increasingly being used in discovery and are now admissible in court. As a result, information that your organization is collecting could be used by (or against) you. In addition, you might be spending money duplicating efforts to mine for data that your log files have already collected for you. It is important for CSOs to understand the wealth of information being collected in their log files, and how they can use that information to its fullest.

A log file is simply a file that makes a record, or a log, of some actions that a computer has performed. Logs on Unix systems tend to be stored in ASCII text files, with one line per log entry. Logs on Windows systems can be stored either in ASCII log files or as "events" inside the Windows Event database managed by the operating system. On either system, log entries typically contain a date and time, the program responsible for creating the log message and a short text description of what actually happened. Log files are traditionally used for debugging. While the information in these files can be corrupted, the majority of the files are accurate.Traditional Use Gets ExpandedThe most common "official" uses for log files include billing, utilization analysis and incident management. For example, a Web-hosting provider might have a program that processes the log files from its Web server to determine how many gigabytes each of its customers transmitted in a month so that they can be billed appropriately. If a hacker starts probing a script for vulnerabilities, those repeated probe attempts will likewise show up in the logs.