Cybersecurity spend: ROI Is the wrong metric

Think about what your network defenders do throughout the day, every day, in the course of getting their jobs done. Can you describe it in one sentence? How would you characterize the thousands of tasks that the InfoSec team fields every day?

For the past few years, my role at Palo Alto Networks has included traveling around the world to talk with board members and C-level executives, and it’s been a fascinating educational experience. Our conversations mostly revolve around cybersecurity strategy, and what I’ve learned is that everybody has a different take on how to defend an organization against cyber adversaries. One question that inevitably comes up is: “How much money should I spend on security?” In an attempt to benchmark and evaluate their own spend, some will ask, “What are other organizations like mine spending?” Others want to know how to calculate the return on investment (ROI) for their security spend.

These questions are common, but they indicate a fundamental misunderstanding about how to evaluate the efficacy of a cybersecurity program, and a misguided approach to resourcing for them. Rather than focus on ROI, I advise executives and board members to focus on network defender first principles.

Network defender first principles

How do you want your network defenders to spend their valuable time? What do you want them to accomplish? What is the 140-character Twitter line that describes the essence of that effort?

The answer to these questions will define what your security first principle is.

The idea of first principles has been around since the early Greek philosopher days. To paraphrase Aristotle, first principles in a designated problem space are atomic. They cannot be broken down any further. They are the building blocks for everything else. They drive every decision you make.

In any discussion about security first principles, I guarantee that your answers will reflect very little, if anything, about what other organizations are doing. Comparing yourself to other organizations, or calculating the return on your security spend, is probably not even in the top 10. If you asked me to define it, I would say my security first principle “Twitter line” is:

Prevent material impact on my organization #FirstPrinciples.

The key word in that phrase is “material.” The number of ugly things cyberattackers could possibly inflict upon your organization in infinite. If you try to get your head around all of them, you’ll become completely overwhelmed. You might convince yourself to punt and just do what the other organizations are doing. You might make your security teams jump through hoops trying to calculate the ROI on the security spend as if somehow you are going to make money by trying to protect your enterprise. You might calculate the total cost of a cyber compromise compared to the amount you spent on security tools. All of these decisions would be misguided.

How do board members identify high-probability cyber risks?

Instead, what you should be demanding from your security team is an identification of the high-probability cyberthreats that would have material impact on your organization in the next one to three years. When I say “high-probability,” I mean the chances are high that a cyber adversary would be successful using a specific attack sequence that would cause the organization material harm. I say one to three years, because it is important that the forecast be time-bound. By narrowing your focus to the most likely threats during a specific time frame, you are positioning your InfoSec team to more effectively identify the threats that pose a likely risk. Once identified, you can then consider how you might adjust your security posture to reduce the risk that a specific attack sequence will be successful.

This is hard to do, but not impossible. Typically, the network defender community is not very good at it. These security leaders need help from the board and the C-staff to clarify their thinking. Typically, most network defenders will develop a list of cyberthreats likely to target an organization. Through careful analysis, they label them as high, medium or low, depending on the circumstances and the organization’s defensive posture. Generally, they will ask the organization’s leadership for funds to defend against more impactful threats; but if you ask them how they developed the rankings, you might not get a precise answer. You might hear something along the lines of, “Well, you know, 25 years’ experience, blah, blah, blah, trust me, blah, blah, blah, cyber is scary, blah, blah, blah, can I have the money please?” This isn’t helpful.

In his book, “How to Measure Anything: Finding the Value of ‘Intangibles’ in Business,” Douglas Hubbard expands Paul Meehl’s concept around clarification chains: “If it matters at all, it is detectable/observable. If it is detectable, it can be detected as an amount (or range of possible amounts). If it can be detected as a range of possible amounts, it can be measured.” In Philip Tetlock’s book “Superforecasting: The Art and Science of Prediction,” Tetlock discusses the absurdity of predicting things without specifying a time frame. He also says that there is no way to hold risk managers responsible for their estimates without having accurate metrics. Using those two risk measurement authorities as a backdrop, it is clear that if network defenders have any hope of preventing material risk to their organization then they must predict risk to business leadership with meaningful metrics. Board members and C-level executives should demand this precision.

A few words to network defenders

While this article has focused on helping board members and C-suite executives understand how to quantify the value of their cybersecurity investment, the InfoSec team may need to assist in the effort. If management is making the mistake of asking IT to justify its cybersecurity budget in terms of ROI, the InfoSec team needs to educate management as to why the ask is wrong and refocus them on the correct one.

Furthermore, when making your argument against focusing on ROI, you need to provide the right data to support your point. Based on my experience, when asked to report on the security readiness of the network, most teams simply provide management with an exhaustive list of every potential threat that could harm the network; the strategy being that, when management sees a list of thousands of potential threats, they’ll agree to any budget out of fear and misunderstanding.

A more effective way to communicate to management about cybersecurity risk is to use business terms and metrics with which they’re familiar. While the C-suite might not understand the different risk level one threat presents over another, its members most certainly will understand the negative impact a successful attack can have on the company’s bottom line (lost revenue, costs to conduct the forensic investigation of the attack and repair any damage caused, customer law suits, etc.). To help you down this path, I recommend two books from the Cybersecurity Canon Project that will enable you to demonstrate to your C-level execs and board of directors how your team evaluates business risk from a cyber adversary: “Measuring and Managing Information Risk: A FAIR Approach,” by Jack Freund and Jack Jones and “How to Measure Anything in Cybersecurity Risk,” by Douglas W. Hubbard and Richard Seiersen. 

If you are basing your cybersecurity spend decisions on what other organizations are spending their money on, you are wasting your time. If you are trying to make your own network defenders justify their defensive posture budget through some revenue metric like ROI, you are not really understanding the problem space. Instead, you should be pushing your security team toward first principles in order to reduce the risk of material impact on your organization in the short term.