Enterprises are Analyzing Lots of Internal Cybersecurity Data

The cybersecurity industry has been talking about the intersection of big data and cybersecurity analytics for years, but is this actually a reality or nothing more than marketing hype? The recently published ESG research report titled, Threat Intelligence and Its Role Within Enterprise Cybersecurity Practices, only reinforces my belief that big data security is tangible today, and enterprises will only double down in the future (note: I am an ESG employee).

As part of the threat intelligence research project, ESG surveyed 304 cybersecurity professionals working at enterprise organizations (i.e. more than 1,000 employees), and asked them which types of internal security data they regularly collect, process, and analyze today. It turns out that around 40% of enterprises collect and analyze 13 different types of cybersecurity data. At the top of the list:

• 52% of enterprise organizations collect, process, and analyze endpoint forensic data. It came as a surprise to me that endpoint forensic data analysis came out on top, but it does makes sense given all of the social engineering attacks of late. Endpoint forensic analysis can help pinpoint specific anomalous system activity, so it is a great complement to network sandboxes and commercial threat intelligence feeds. Some organizations do endpoint forensics on an ad-hoc basis using open source tools, but many are adopting commercial tool from vendors like Bit9 + Carbon Black, Guidance Software, or RSA (ECAT).
• 48% of enterprise organizations monitor sensitive data access and usage. This type of monitoring has become a best practice as a countermeasure to APTs and data exfiltration. It also aligns with the recent market renaissance of data security initiatives using DLP/eDRM tools from companies like Digital Guardian, Informatica, Ionic Security, Symantec, and Varonis.
• 46% of enterprise organizations collect, process, and analyze endpoint/server profiling data. While endpoint forensics details machine activities (i.e. file downloads, registry changes, in-memory processes), endpoint and server profiling monitors the state of each device – configuration settings, hardware configurations, installed software patches, etc. Endpoint/server profiling acts as a real-time asset management repository for risk management, allowing the SOC team to react to changing threats and vulnerabilities with tools from vendors like ForeScout, Great Bay Software, Promisec, Pulse Secure, and Tanium.
• 41% of enterprise organizations collect, process, and analyze network packet capture data. Network forensic data is a perfect match to endpoint forensic data analysis. If you collect and analyze what’s happening on the network and endpoints, you certainly have the right information for connecting the cybersecurity dots – somewhere in the haystack anyway. PCAP is the domain of companies like Arbor Networks, Blue Coat (Solera), Lancope, and LogRhythm.

It’s also worth noting that 35% of enterprises plan to collect “significantly more” internal cybersecurity data over the next 12 to 24 months, so big data security analytics initiatives will continue to grow in data capacity and complexity. I expect more big data technologies and data scientists to elbow their way into this market as this happens. Heck, we’ve already seen examples of this with Splunk buying Caspida, while vendors like Cloudera, Hortonworks, and Sqrrl add cybersecurity algorithms to their platforms. 

All of this data collection, processing, and analysis seems like a good thing, for as Sun Tzu stated, “If you know the enemy and know yourself, you need not fear the results of a hundred battles.” Following this advice assumes that we can turn cybersecurity data into actual knowledge, actions, and countermeasures. This is the real challenge facing the enterprise cybersecurity community.