• United States




SIEMs sometimes suck

Jan 11, 20174 mins
Network SecuritySecurity

SIEMs can provide a ton of value and are central to many security teams, but then again, SIEMs sometimes suck.

Security Information and Event Management (SIEM) solutions provide great value. I even like some of the general-purpose log management solutions that can be used for IT management and the placation of auditors. Early in my career I cut my teeth at ArcSight so I’ve been around SIEMs for a while. SIEMs can provide a ton of value and are central to many security teams, but then again, SIEMs sometimes suck.

There are many issues people take up with SIEMs, but one of the biggest problems with SIEMs is that to be useful they are dependent on the data they are being sent and variables related to that data such as volume, velocity and variety. Common questions related to SIEMs are: 

  • Is my SIEM getting enough useful data?
  • Is my data coming at a timely cadence?
  • Is my data formatted correctly so it can be processed?
  • Are my timestamps accurate?
  • Are my sources generating accurate logs, alerts?

But let’s pretend that all the above issues have been resolved and your SIEM is purring along. You are collecting raw data, generating metadata, firing rules, launching alerts, etc. But let’s focus in on “firing rules” because that’s when things really start to suck.

When a rule fires within your SIEM it’s generally a reaction to a trigger associated with some level of temporal or volumetric aggregation, correlation or pattern discovery. The SIEM must be told what to look for and that’s a problem because you might have built hundreds of rules in your SIEM to look for specific attacks and attack patterns. But you might not be able to validate if any of those rules are going to fire based on the type of data that you are receiving into your SIEM and the design of your SIEM’s rules.

So, the real sucky part of your SIEM strategy is the “hope” that you are developing rules that actually work. The rules must be validated and more to the point those rules must be validated based on your specific security infrastructure blocking, detecting and allowing real malicious traffic.

Rules that work on a SIEM for company A might not work for company B even if you are using the same SIEM with the same firewalls, IPS and related security solutions because of configuration differences.

Further, simulated attack recreations, where the attacks aren’t the real thing, provide little value beyond illustrating how your security infrastructure might respond to a simulated attack recreation as opposed to a real attack, as simulations can miss key attack functionality. It’s kind of like doing a crash test on a car and instead of crashing the car into a real brick wall to see how well the airbags, anti-lock brakes and seat belts work, you crash it into a polystyrene wall painted red with a sign reading “I’m a wall.” It just isn’t the same thing. This is why the MythBusters used Buster and real walls and why you need to use real attacks, but use them safely, not like Buster – RIP Buster. 

What to do

To help address these SIEM issues solutions are needed that can:

  • Safely execute real attack traffic within your production network.
  • Determine that your incident prevention solutions are preventing the attacks.
  • Conclude that your incident detection solutions are detecting the attacks.
  • Verify that your SIEM is receiving information related to the attacks.
  • Validate that your SIEM is firing rules based on the received attack information.
  • Generate the rules that your SIEM can use, based on your security controls and architecture, to detect future attacks.
  • Continuously evaluate your SIEM’s readiness to avoid defensive regression.

By leveraging these capabilities your SIEM rules won’t be based on “hope” but rather empiric evidence. You will be sure you are getting the right source data and preforming the relevant correlations necessary to trigger on real attacks in your environment. In short, your SIEM won’t suck.

More on SIEM:


Over the last two decades Brian Contos helped build some of the most successful and disruptive cybersecurity companies in the world. He is a published author and proven business leader.

After getting his start in security with the Defense Information Systems Agency (DISA) and later Bell Labs, Brian began the process of building security startups and taking multiple companies through successful IPOs and acquisitions including: Riptech, ArcSight, Imperva, McAfee and Solera Networks. Brian has worked in over 50 countries across six continents and is a fellow with the Ponemon Institute and ICIT.

The opinions expressed in this blog are those of Brian Contos and do not necessarily represent those of IDG Communications Inc. or its parent, subsidiary or affiliated companies.