RSA panel to ponder risks, benefits of Big Data

I’ll be moderating a panel at RSA about the security pros and cons of Big Data.

The panel thus far includes Adam O’Donnell from Sourcefire, Andy Jaquith from Perimeter Security, Rich Mogull from Securosis and John Adams from Twitter. We’re scheduled for Feb. 28 at 3:50 p.m San Francisco time.

Please chew on this pros-cons list and send me any questions or comments you’d like us to address. The list, by the way, was put together by some friends at Sourcefire who initially approached me about doing the panel…

Pros:

•Security is often about detecting anomalies, and to do so, you need to have a full spectrum view that you typically can only get if you have enough data to know what constitutes “normal” versus “abnormal”.

•The goal with many information security solutions is to translate “back office intelligence” into “customer facing protection”. In recent years, the amount of back-office intelligence security firms are dealing with has grown tremendously (e.g., growth of malware samples, large volumes of sensor data, etc.). Big data techniques lend themselves nicely to this domain.

•To make the most accurate (security) decisions, we need to take advantage of all the intelligence available to us – from sensors, logs, user activity, etc. Big data techniques can be used to extract the most value from this wealth of information.

•Big data techniques are also useful in doing more broad visualization of security-related metrics. Having such a big picture understanding can help identify root causes to problems. In contrast, many “traditional” approaches only address symptoms rather than causes.

•Big data techniques can lead to entirely new sets of security capabilities. For example, in Sourcefire’s case, retrospective threat detection fundamentally leverages big data techniques. We are likely just scratching the surface here, and there are a wealth of new opportunities waiting to be uncovered.

Cons:

•While there has been a rapid proliferation of “big data” technologies out there, not all of them are well baked enough to be used in production environments

•Security decision-making needs to be rapid, and that does not always align with the batch-oriented processing of large data sets.

•There are no one-size fits all big data technologies. You have to understand both the problem you are trying to solve and the technology you are thinking of leveraging to solve it. If you aren’t sufficiently familiar with one or the other, there is a good chance your approach will ultimately prove fruitless.

•When you have a powerful hammer, everything starts to look like a nail. Big data techniques are powerful, but not every security-related problem requires them, nor can they magically solve every problem that comes up. Instead, it’s important to apply domain expertise and common sense first.

•Before focusing on “big data”, focus on “good data”. Many people try to apply sophisticated data mining techniques, but on data that might be dubious or otherwise poorly collected. For all their merits, data mining techniques are very much “garbage-in-garbage-out”.

Let’s have some fun with this and, hopefully, offer the community some actionable feedback in the process. Thanks!

–Bill Brenner

one-stop view of latest business threats. We created it for you! Bookmark it! Use it!

CSO’s Daily Dashboard gives you a

Get your morning news fix with the daily Salted Hash e-newsletter!