Home » Data Protection » Network Security

Another Look at Log Files

These long-standing logs can help you monitor your networks and employees. So before you invest in a new kind of data collection system, review your log files.

One way to minimize the chances of log files being maliciously modified—and to increase the chances that your logs will hold up in court—is to store them on a special "log server" to which access is generally restricted. Setting up a system such as this is easy with Unix; the industry standard "syslog" logging utility has supported remote logging since the early 1980s. You can either set up a Unix workstation with a lot of disk space as your log server, or you can purchase a specially built logging appliance. In the Windows world, there are a number of remote logging systems available.

Determine Your Intentions

You must decide ahead of time whether you intend to use your logs as video recorders or burglar alarms. If your logs are primarily a recording system, then you will consult them only when evidence of wrongdoing arises from some other channelfor example, if your CEO receives a death threat. In that case, you'd examine all of the logs at your disposal to see who was in the building, where they were located, and what they were doing.

If you want to use your logs as an alarm system, you'll need to have a person or an automated process that regularly scans the logs for something noteworthy. This can be a challenge, because you often don't know exactly what you are looking for. The ideal log analysis tool would alert you to unauthorized or unusual activity. But how does a computer know what's unauthorized or unusual?

Accuracy Is the Goal

Not surprisingly, designing systems that can automatically recognize the unusual has been a hot area of research for many years. It is an easy problem to solve, but solutions are difficult to implement well. As with all recognition problems, the issue is accuracy: Make the system very sensitive, and you will get many false positives; make the system less sensitive, and important events will slip right through the cracks.

My favorite technique to analyze log files is to have filters that do not recognize unusual events, but instead recognize usual ones. Run these "negative" filters, and everything that's leftover is what you should focus your analysis on. If you see too many "normal" events that should have been filtered out, you respond by writing yet another filter. This is also the approach that Ranum recommends.

Vigilance Is Key

Making a negative filter work in practice, though, is hard: Even when it works properly, you end up with a system that is constantly trying to attract your attention. Put the reports on a webpage, and you'll forget to check them. Arrange to have them sent by e-mail, and you'll quickly learn to hit the delete button after a fast and insufficient scan. Ultimately, the only solution for this problem is vigilance, proactive auditing and penetration testing. If the auditor's attempts to break into your network aren't picked up by your logging system, then it's time to revamp the system, replace the people who are using it, or delete all of your logs and use the disk space for something more productive.