The Intersection of Security Intelligence and Big Data Analytics

It’s official, the security industry has jumped on the “big data” bandwagon with both feet. How do I know? Well, I’m participating in a panel discussion on this topic at RSA and I believe there are 2 other sessions on the topic. I guess anyone headed to San Francisco later this month should be prepared to get a big dose of big data.While lots of the rhetoric around RSA will be just that, I think this particular dialogue is worthwhile. Today’s security intelligence requirements demand massive amounts of data collection, retention, and analysis because:• New threats are bypassing old monitoring tools. In spite of huge investments in the IT security equivalent of locks, surveillance cameras, and guard dogs, our adversaries have figured out how to penetrate the network, blend into normal network and host behavior patterns, and then find and steal our valuable data. These stealthy attacks fly under our current radar systems so it is logical to conclude that we need better ones. • Security intelligence demands more data. Early SIEMs collected event and log data then steadily added other data sources like NetFlow, packet capture, Database Activity Monitoring (DAM), Identity and Access Management (IAM), etc. Large enterprises now regularly collect gigabytes or even terabytes of data for security intelligence, investigations, and forensics. Many existing tools can’t meet these scalability needs.• CISOs need real-time risk management dashboards. Assessing enterprise security posture often involves manual processes and discrete reports. What’s needed is a true dashboard with up-to-date information on assets, configurations, vulnerabilities, threats, and behavior monitoring. There are some compliance tools in the market designed for these requirements but most have a long way to go. Our existing tools aren’t providing us with the security intelligence needed so in essence, we are “flying blind” leading to a substantially increase in IT risk. As Sun Tzu said, “if ignorant both of your enemy and yourself, you are certainly in peril.” From a supply-side perspective, security vendors certainly see the need for big data analytics capabilities in security intelligence. IBM bought Q1 Radar and will certainly weave in some big data technology like InfoSphere and i2. Same goes for RSA Security with enVision, Archer, NetWitness, and Greenplum. HP’s also anticipated the security intelligence/big data analytics intersection when it bought ArcSight, as did McAfee when it scooped up Nitro Security. Independents like LogRhythm, Red Lambda, and Splunk are also active here. The bigger challenge will come from the demand side. CISOs know that their existing portfolio of security intelligence tools is inadequate, but they don’t want to repeat past mistakes by buying a bunch of tactical solutions. The next-generation of security intelligence tools must provide big data analytics intelligence, multi-terabyte scale, and out-of-the-box value. No one wants to sign up for 2-years of service for security intelligence application customization or hire a bunch of quantitative data experts to work with security analysts. Allow me to indulge in one other quick point here. While “big data” will intersect with security intelligence, the actual “big data” technology aspects are irrelevant. CISOs need the analytics capabilities but really don’t care what’s under the hood. Let’s focus on data analysis and situational awareness and avoid a debate about OLAP, Massively-Parallel Processing (MPP), and Hadoop.