When it comes to security, it’s the data, stupid

In an election year, particularly one in which we’re all bracing for a downturn, the 1992 Clinton campaign’s famous catchphrase “It’s the economy, stupid!” can’t help but come to mind. Apply that same commonsense thinking to computer security and you get: “It’s the data, stupid!”

We suffer from a dearth of data and quality analytics on how we’re exploited and compromised. We know most of the likely root causes: unpatched software, social engineering, eavesdropping, password cracking/guessing, data leaks, misconfiguration issues, denial of service, insider threats, zero days, and so on. But we lack good metrics on how often they occur inside our environment.

We understand that we’re getting exploited by malware — we may even have the number of detected and removed malware programs in a given period — but we probably have little data on how many times social engineering let a bad guy in. We may know every unpatched program in our environment, but probably not which one is letting in the most damage. We simply don’t know how each threat ranks against each other.

The upshot is that we respond to crisis events and gut feelings. It’s about time we started to mature our defenses by asking for data, good metrics, better reports, and ultimately accountability. If you really think about it, our lack of data should be embarrassing to us. How can any organization perform risk assessment when the threats and risks haven’t been quantified?

Start collecting data now

I spent the first three decades of my career wondering why all the wonderful computer security defense tactics, strategies, and tools didn’t work to make our computers safer for work and play. I’ve decided that I’m going to dedicate the last two decades of my career forcing IT security environments to think about and collect more data.

Every other part of the organization runs on data, from HR to finance to building maintenance. I can probably ask any janitor in any building how many rolls of toilet paper are used in their building each week and receive an accurate answer. But ask any IT security person what their company’s biggest security threat is, backed by data, and you’ll usually get a puzzled stare.

The Holy Grail of IT security defense data is the number of times a particular root cause exploit was used to successfully compromise your enterprise. If you got a report that said something like this:

You might be able to start to focus everyone on the risks that matter the most. Of course you’d need to take root cause exploit occurrences and multiply them by the damaged they caused to get a better list, but even with this list alone, you’d have actual data from which to work smarter.

The idea of ranked data needs need to become pervasive through IT security in every organization. Don’t bring me an unranked list of anything. I want ranked relevancy.

Want me to start fixing vulnerabilities? Give me the data. And I don’t mean the number of vulnerabilities. That number means little. Also, don’t tell me it’s critical. Nearly everything in our world is critical. I think three-fourths of the vulnerabilities on CVE lists are critical. No, what I want to hear is how much X vulnerability is successfully exploited in the environment, especially compared to other vulnerabilities.

I may have 1,000 unpatched Windows servers, but if they are being exploited more through unpatched Compaq Insight Manager, then I need to focus on the latter before the former.

Some readers will tell me it’s impossible to get this sort of data. In some cases it may be difficult, but seldom impossible. I know we can collect far more data than we are gathering today. In most cases we aren’t even trying. Sometimes a “best effort” gives us enough to get started.

Even more important is to establish a culture where data is king. Gut feeling is fine. But back it up with data before you act on it.

Pitching to management

Data is the language of CIOs and CISOs. How can you run to a CIO or CISO asking for money to fund security technology or best practices without risk-relevancy data? By the time you step into that office you should have hard data to support your bullet points.

Imagine walking up to your CISO and saying, “We identified X root cause as behind 49 percent of our successful exploits. It’s our No. 1 problem. By reducing this single cause we can get rid of nearly 50 percent of our current computer security risk. I’d like to put together a project team to explore how we can best mitigate this issue. Here’s the data and here’s how we will measure future success.”

I can’t imagine a CISO not being knocked out by such an approach incorporating real data, focus, and accountability.

It’s a myth that management isn’t giving us the resources we need to do our jobs better. The reality is that we haven’t been providing the background data to make the kind of well-supported arguments CIOs and CISOs are accustomed to hearing.

How to get started

What data you start to collect depends upon many of factors, beginning with what data you already collect and where the gaps are. In general, a good data event that ends up creating security alerts should contain the following attributes:

• High likelihood that occurrence indicates unauthorized activity
• Either a single occurrence or an unexpectedly large number of events in a given time period indicates a high chance of maliciousness
• Low number of false positives
• An alert occurrence always results in an investigative/forensics response

If you haven’t guessed by now, I’ve become a data warrior. I’m already meeting with CIOs, CISOs, and the rest of my team members to ask them what data they want to see that they don’t see today. I’m meeting with my data people to find out what they have and what they think we might need. A data-driven computer security defense is a new paradigm. We’re going to need all the help we can get.

Next time someone brings you an unranked list of things to do or fix, ask about the relevancy and data. Make it a habit.