Are you afraid to measure what matters in security?

Measuring what matters in security takes guts and the ability to truly understand and guide the business

A few months ago, I got approached to review an effort to build a dashboard to report on security metrics. Part of a larger effort to demonstrate the value of security, they were eager to show off their work.

They sent me a spreadsheet. It had a pivot table. It looked pretty. It made no sense to me.

We scheduled a call for them to walk me through it. They explained that they tracked vulnerabilities against five business servers. Combined, they found over 1400 "high priority" vulnerabilities.

As I processed what they shared, I asked a few questions:

  1. Who selected the servers - you or the business? We did.
  2. Are those servers important to the business? We don't know, but we think they are.
  3. As a business leader, what does it mean to have 1400 "high priority" vulnerabilities? Awkward silence.
  4. How long would it take to fix? How much would it cost? Good question. We don't know. Not sure how to figure it out.
  5. What would fixing this potentially break or disrupt? We have no idea.
  6. Where does this fit in the scheme of everything else clamoring for my attention, resources, and budget? Are you doing anything in security that reduces the real impact of these vulnerabilities? We're not sure.
  7. How does this compare to the previous months? Am I getting better, worse? Is my actual risk increasing? We have no idea, we're not recording the trend.

It ended up a pretty short call. I could hear the dejection in their voices. My goal wasn't to be negative or insult them. I just asked the questions that came to me as a result of what they shared and my experience on the other side of the table.

In their defense, they thought they were measuring what matters to security. Unfortunately, they focused on what they could count. And that didn't matter to the business, or to security.

They were afraid to go to the business and find out what was important to them. They wanted to have something to show first, in an effort to prove their value.

Lucky for them, they didn't lead with that spreadsheet.

The lesson: measuring what matters in security means understanding and measuring what matters to the business.

Without question, security is essential to the success of modern  organizations. The challenge is understanding the role of security to connect with and support the business.

In the end, what matters in security is what matters to the business, but with a security lens applied.

The fear of measuring what matters, in numbers

Tripwire recently surveyed over 1000 people, and asked “Why don’t you create metrics that are well understood by senior executives?”:

  • 59 percent said the information is too technical to be understood by non-technical management
  • 48 percent said pressing issues take precedence
  • 40 percent said they only communicate with executives when there is an actual security incident
  • 35 percent said it takes too much time and resources to prepare and report metrics to senior executives
  • 18 percent said senior executives are not interested in the information

These findings suggest that we're afraid to measure what matters. First step: getting over the fear of measuring what matters.

Sometimes the business isn't sure, either

To be fair, strolling into the office of a business executive and asking, "so, what should we measure?" doesn't generally end with a prepared, concise list.

Measuring what matters is tricky, regardless of the situation. That's where knowing the three steps that work in any situation are helpful.

It's an opportunity to demonstrate value to the business. Work through the process and come to a mutual understanding. Then use your experience in security to figure out what elements actually do matter to the business.

Sometimes it means trying a few different things to see what works. Capture what you can, show people what's possible, set the context and explain why (you think) it matters.

Then discuss until the right solution is evident. At least for now. Measuring what matters is an evolution. Getting started and supporting the business is key. Capturing these three elements makes sure the program is poised to evolve properly.

When we measure what matters, we have the ability to learn. That allows us to provide better security services as well as advance the goals of the organization.

A few questions to help measure what matters

When trying to figure it out, use these three questions to get started:

  • What am I measuring? And why?
  • What does this measurement tell me?
  • If presented to an executive, does it allow them to make better decisions? A better decision is one that increases the value of the company.

Check out some recent articles on measuring what matters for additional insights, questions, and guidance:

By engaging the business in productive, structured discussions and taking time to learn what matters to them, we're able to measure what matters to security. 

Copyright © 2013 IDG Communications, Inc.

22 cybersecurity myths organizations need to stop believing in 2022