The Big Data Security Analytics Trifecta

Like many of my industry peers, I’ve been writing and speaking a lot about big data security analytics. The general hypothesis is that status quo security processes and technologies no longer provide adequate protection against voluminous, sophisticated, and targeted threats, so we need better security analytics to understand what’s happening in real-time. With improved situational awareness, we can accelerate incident detection and response.Okay, this makes sense as a theoretical concept, but what exactly is the “big data” behind big data security analytics. In my mind, big data security analytics solutions will reach their true potential when they collect, process, analyze, and correlate data related to:1. Network behavior. We’ve been travelling down this road for a long time but much work remains. To really understand network behavior, you have to know about devices, applications, protocols, IP addresses, users, typical behavior, etc. We used to collect security device and network logs to figure this out. Now we are collecting ever-growing volumes of other network data including NetFlow, IP packet capture, application profiling, and we are also likely to see a further blending of security data and network operations data in this realm (Think Click Security, Lancope, NetWitness, Solera Networks, etc.). Still, a 10gb network pipe moves approximately 15 million packets per second so security analytics at the network level will continue to be a challenge. The key success factors to me are context (i.e. what’s going on “up the stack”) and algorithms (i.e. detecting anomalous network behavior “up and down the stack” accurately in real time).2. Security intelligence. Researchers have always set up network honeypots to look at threats “in the wild,” but they used this data to create antivirus signatures or author research reports. This pattern changed over the past few years as security vendors like Blue Coat, Kaspersky Lab, Trend Micro, and Websense integrated on-premise security products with cloud-based intelligence to bridge the gap between detection and prevention. Big data security analytics platforms are joining the party now, consuming real-time threat intelligence that can then be correlated with data gathered internally for better decision making. To make this process as efficient as possible, threat intelligence vendors should support the Structured Threat Information Expression (STIX) and Threat Information Exchange (TAXII) standards for threat data enumeration, syntax, and transport protocols being developed by DHS and Mitre.3. Network state. Okay, this is the ugliest of the triplets. By network state I mean the assets connected to the network, their current configurations, their histories, status changes, etc. Oh, and we also need to know which users are behind these devices. Yes, I realize we already collect most of this data but we do so through an army of disparate management tools. Try getting a complete view – it’s a mess at many organizations. In a perfect world we need more than simple information about individual devices; we need a complete picture of network connections in order to understand and address systemic risk across all of IT. Now think about analyzing these three types of data in concert: The network displays anomalous behavior that is correlated against recent threat intelligence. It appears that this may indicate that a Windows system has been compromised by a new strain of malware aimed at a particular version of the Firefox browser. The exploit uses a compromised URL and JavaScript to deliver its exploit. Analysts can then determine which systems have Firefox installed, which users are apt to use Firefox rather than Chrome or IE, which have JavaScript enabled, which VLANs these users are on, which departments they are in, etc. Whether this analysis is automated or manual, access to all of this data will certainly help speed things along. I know this is a simple example and organizations with strong security could probably figure this out today. True, but how long would it take them, how many people would be required, and how many tools might be involved? When big data security analytics encompasses network behavior, state, and security intelligence, it will make this process more efficient, effective and a whole lot easier. Note: For those interested in more information about the big data security analytics landscape, go to the ESG web site and download the ESG Market Landscape Report, “The Evolution of Big Data Security Analytics.”