SANS survey: IT spending too little time analyzing logs for security clues

Relatively few organizations are making good use of gobs of log data they collect for purposes such as identifying attackers, according to a survey of 600-plus IT professionals by security outfit SANS.

According to the SANS Analyst Program survey on log and event management, “Sorting through the Noise,” 22% of respondents use a security information and event manager (SIEM) to collect and analyze data, while 58% use log-management systems, and the remainder rely on other means. Most respondents said one of the main reasons to collect logs is for the purpose of regulatory compliance, though 9% discounted the importance of that.

BACKGROUND: Meeting data privacy, residency and security requirements in the cloud

As in previous years that SANS has done this type of survey, virtually all the respondents said that “detecting and tracking suspicious behavior was important.” But according to SANS, there’s evidence that insufficient time is being spent in actually analyzing the collected log data.

“The data suggests that respondents are having difficulty separating normal traffic from suspicious traffic,” said Jerry Shenk, author of the SANS report www.sans.org/reading_room/analysts_program. “They need advanced correlation and analysis capabilities to shut out the noise and get the actionable information they need. But first they need to get more familiar with their logs and baseline what is normal.”

The key issue in log analysis was cited to be “indication of key events from normal background activity” and “correlation of information from multiple sources.” According to the survey, organizations are typically collecting log data from Windows and Unix-type servers, security devices, network equipment such as switches and routers, intrusion-detection systems and anti-virus and other security applications, and virtualized servers and hypervisors, as well as desktops and laptops.

Organizations want to detect suspicious activity but when the IT professionals were asked how much time they normally spend on log-data analysis, the largest group (35%) replied, “none to a few hours per week.” As for the rest, 18% didn’t know, 11% said one day per week, 2% outsourced this task to a managed security service provider, and 24% defined it as “integrated into normal workflow.” The SANS survey report, which notes analysis time overall actually seems down from last year, noted that about 50% of the smaller organizations spent zero to just a few hours analyzing logs.

Overall, “that is really not very much time spent getting familiar with logs,” the SANS report states. “Given the advanced threats they are struggling with, we would have expected the time organizations spend on log analysis to increase, not decrease. We cannot stress enough that the best way for organizations to quickly detect abnormalities is to gain understanding of their baseline or ‘normal’ activity by reviewing/analyzing log data on a regular basis.”

The SANS report points out that “SIEM-type tools, including log management tools with analysis and reporting options, will help organize and identify patterns and activities that are generally recognized as indicators of problems. Yet, 58% of organizations are not anywhere close to that level of automation.”

At the same time, the SANS report emphasized that automated tools cannot be viewed as a complete substitute for the people who are log analysts who develop a “sixth sense” about traffic anomalies and security because they spend some time every day looking at log data.

When it came to defining difficulties, trying to detect so-called “advanced persistent threat” attacks–APTS being the term to describe stealthy intrusions into the network to steal sensitive information—ranked as the toughest problem, according to the survey, with 85% this year reporting this as an issue in comparison with 65% last year.

Ellen Messmer is senior editor at Network World, an IDG publication and website, where she covers news and technology trends related to information security.

Read more about wide area network in Network World’s Wide Area Network section.