Information overload, SIEM version

It’s been over a year since I last wrote about my security information and event management (SIEM) platform — and a lot has happened since then. Back then, I wrote, “Now that my SIEM has been in operation for several months, I’ve become completely dependent on it, not only for security monitoring, but also for overall awareness of my network.”

Since that time, I’ve only become more dependent on my SIEM for keeping track of all the alerts being generated by my various security information, alert and log sources. At last count, I had 21 different systems feeding data into my SIEM, including intrusion-detection sensors on the network, malware detection on the network and individual computers, firewall logs, network device logs and flow data, and server logs. All this information has given me unprecedented visibility into threats on my network — and now is the right time to have that visibility.

Looking at all the data breaches in the news over the last year (including the top 20 breaches I wrote about last month), one thing they all have in common is a lack of timely detection. In fact, most of the victims had no idea they were breached until the U.S. government’s three-letter-agency watchdogs notified them. The attackers operated undiscovered for months on those networks before they were discovered. It’s my belief that a good SIEM would have alerted those organizations to the attackers’ activities, such as phishing, malware exploits, unauthorized remote access and data exfiltration. Certainly, my SIEM would do so.

[ SIEM: Dead or alive? ]

How can I have so much confidence in my SIEM? Because I use it every day, and it reliably alerts me to all of those threats. When I last talked about my SIEM, I mentioned that I was looking into third-party services to monitor it as well. Since then, I’ve actually gone through three different monitoring services. The first two were disappointments, but the third is doing a really great job of escalating the important alerts while tuning out the false positives and less important data. I find threats on my network every day — usually malware, most often caused by poisoned Web searches that employees stumble across while doing personal searching. The poisoned search results usually fly right through the employees’ browsers without their knowledge or interaction, resulting in infections that set off my alarms. When that happens, one of my team members pays a visit to the victim, confiscating the hard drive and offering advice on how to avoid infections in the future.

I have a good, reliable SIEM technology that pays dividends every day. So what could go wrong?

Too much information, that’s what. Not coming out of the SIEM, but going into it. I have so much data pouring into my SIEM that it’s actually overloading the network. My SIEM is fine — it’s built to handle massive amounts of data flow — but the network bandwidth itself is becoming saturated by all the alerts and logs. Not only does this lead to complaints from our network engineer, but unreliable service as well. For example, some of the data flowing into my SIEM is in the form of “spans” from network routers and switches. These spans duplicate all of the traffic flowing inside my company’s network, which is very useful for SIEM analysis. But when the network gets bogged down from too much traffic, the routers and switches automatically cut off the spans so they can focus on delivering network traffic. When that happens, my SIEM goes blind.

What I’m planning to do about this situation is to offload some of the traffic from the routers and switches onto a specialized data delivery device. The device I’m looking at is designed to sit on the network and mirror the network traffic to my SIEM, so the routers and switches don’t have to. It can also take log and alert data from some of my other sources and carry them directly to my SIEM, cutting down on network bandwidth.

So while I now have too much of a good thing, fortunately the state of security technology has caught up to the problem. If all goes as planned, I can simply drop in the new device and hook it up to my SIEM without any trouble. Then I can add even more data to what I’m already monitoring.

This week’s journal is written by a real security manager, “J.F. Rice,” whose name and employer have been disguised for obvious reasons. Contact him at jf.rice@engineer.com.

Join in

Click here for more security articles.