How to engage with the C-Suite on cyber risk management, part 4

Creating metrics to indicate risk.

businessmen with umbrellas risk protected storm

In part 3 of our metrics series, we discussed we how KRIs help identify risks while KPIs help us measure them. In this, our final article in the series, we’ll build on this knowledge to create metrics based on our four-stage model for qualifying risks and threats that we introduced in part 2. By making your way through our sequential process, you’ll be able to extract performance indicators that can enlighten your executives on security risks and management.

security funnel Chris Hodson

Figure 1. Four-stage model for qualifying risks and threats.

Keep in mind that the metrics that come from the four-stage process should be kept within the infosec team, and used to inform future KRIs. Why? Because the results that are generated by these steps are often in jargon and tech-speak that the board won’t understand – like this:

“We’ve seen a persistent 200 gbps DDoS attack targeting the external interface of our DMZ.”

“Our sandbox devices have been evaded five times through malware with a sophisticated form of payload obfuscation.”

These are not risks: they are threats. If a threat cannot exploit a vulnerability, who cares? If the vulnerability doesn’t impact the confidentiality, integrity or availability of data, so what?

For example, if we’re performing our Business Impact Assessments (BIAs) and we establish a maximum tolerable downtime, it’s the KPIs from our systems that keep us honest. If we have a KRI associated with “shadow IT consumption and data loss”, it’s our KPIs around the percentage of unapproved applications deployed across the estate that provide proof points. In summary:

  • Business strategy tells you the direction the company is going in and how it will get there
  • BIAs uncover the important assets necessary to support the strategy
  • KRIs identify areas of risk which could prohibit a company achieving objectives
  • KPIs measure and keep you honest

Sound confusing?  I hope not – check out Figure 2 where I represented where and how business goals are mapped into risk and performance indicators:

security metrics wheel Chris Hodson

Figure 2. Business Initiatives translate into Indicators.

Step 1: Qualify assets

We cannot apply the same level of security to everything (to do so would be very expensive and may increase friction in the user experience, which is unnecessary). We will apply stronger controls to systems and apps that contain stuff/data that we care about it.

To assess if we care about stuff/data, we leverage the BIA and data classification framework we discussed in above. This is where business continuity and security can align. In your BIAs, define location and technology stack inventory information for all systems that house confidential data, employee records, or systems where the confidentiality, integrity and/or availability requirements are “Critical/High.” Prioritize these first. The outputs of your BIA will give you pertinent information regarding the volume and sensitivity of data assets in your estate. The methodology varies but outputs generally include:

  • Knowing how long you can afford to be without the data in question: Recovery Time Objective (RTO)
  • Knowing how much data you can afford to lose in the event of a compromise: Recovery Point Objective
  • Classification Requirements for Information: Confidentiality, Integrity and Availability.

When you are qualifying these assets, visibility is of paramount importance. We cannot protect what we cannot see. Challenges with visibility stem from four contemporary challenges:

  • The rise of TLS traffic
  • Multiple device types connecting to our network
  • Cloud application consumption
  • Increased regulatory and legal pressure

Let’s say you find 275 laptops in your environment that you don’t know anything about. There may be a legitimate business justification for these unapproved devices. If so, then you can build an argument for investing in a suitable control (which we discuss in step 4 below).

All too often, we exclusively think of an ‘asset’ as a physical device (laptops, phones); data are an asset too; Since we’re soon going to be forced to notify a supervisory authority within 72 hours of a known breach, a solid metric here would be a blue team exercise associated with some incident response scenarios which would provide stakeholder confidence that expedient report can be achieved:

  • How long did it take to penetrate the environment?
  • How long to discover data being ex-filtrated?
  • How long before we had a public statement and briefed relevant board members?

Infosec management has to be a holistic exercise, an imperative consideration being to secure your supply chain: Are you comfortable knowing who is connecting to your network? Let’s build a picture of who is connecting, using metrics associated with vendors who have passed a third-party due diligence process, and who have ISO 27001 certification.

  • How many external repositories house personal data?
  • How many suppliers have completed your vendor and supplier readiness questionnaire?
  • % of suppliers who are PCI-DSS and ISO-27001 certified.

Reports suggest that 75 percent of the web will be encrypted by 2019. Given this, if your organization is still waiting to deploy TLS interception, perhaps we should look to get the visibility heat map flashing red! The total volume of web-traffic encrypted-v-unencrypted through Web and Email Gateways is always a good starting point to justify the need for TLS interception. If you need some assistance with your business case, check out some  produced that shows the rise of threats in SSL.

Step 2: Profile threats

At this point, we know what we are protecting and we know who is connecting and from which devices. Next, we need to better understand who is attacking us. In an ideal world, the security professional would provide a percentage likelihood of a data breach, but in reality, cyber security presents a volume of variables that cannot be measured as absolutes.

For example:

  • Attribution of threat actor location is onerous due to anonymizing services and botnets.
  • Security control strength and application consistency varies across organizations.
  • Forensic breach analysis information is not publicly provided.

That said, we can generally break down threat actors into two categories. We then develop metrics associated with the ‘tools, techniques and procedures’ (TTPs) that actor actor exhibits:


Insiders, third parties, admins who make a mistake (who could forget the Amazon S3 debacle?) A solid metric here is your recovery point objective and recovery time objective – how long can you afford for a system to be offline?  Which threat events could cause such an outage?


Who is attacking us? How well funded are they? Are they committed and capable? Are you able to review previous incident reports and identify the tools, techniques and procedures used? Are others in your industry being targeted by this actor? Are indicators of compromise (IPS signatures, file hashes, PCAPs) available which help identify malicious activity?

There is a lot of talk at the moment regarding threat intelligence, and understandably so. Is your organization pulling in threat intelligence information from open source or commercial providers? Are you participating in industry working groups? It is my view that no single organization can achieve a satisfactory understand of the threat landscape, companies must collaborate. It might feel like a truism but with the attackers only needing to be right once and the customer needing to be right all the time, we need all the help we can get. In any walk of life, collaboration and co-operation is almost impossible if we are speaking different languages; this has long been an issue with cyber information sharing. This series isn’t intended to deep-dive into cyber threat intelligence but I recommend that any organization interested in obtaining the most thorough picture of the cyber threat landscape as possible, looks to obtain feeds in a normalized fashion. The Structured Threat Information Expression (STIX) provides a standardized, machine-readable language for threat information exchange and follows a number of the language components I cover in this series (Actors, Events, Vulnerabilities, etc).

1 2 Page 1
Page 1 of 2
Get the best of CSO ... delivered. Sign up for our FREE email newsletters!