Home » Data Protection » Network Security

A Few Good Information Security Metrics

Information security metrics don't have to rely on heavy-duty math to be effective. They also don't have to be dumbed down.

By Scott Berinato

July 01, 2005 — CSO —

Metrics have a bad rep. Mention metrics to a CISO and immediately his thoughts may well turn to sigmas, standard deviations and, probably, probability. To many, metrics equals statistics.

There’s no denying that proven economic principles can—and should—be applied to information security investments. At the same time, a bumper crop of valuable metrics exist that don’t require classes on Nobel Prize-winning theories or a working knowledge of the Greek alphabet. You’ve actually already sowed the seeds of these less dense but equally valuable metrics. They’re sitting in your log files, on your network, in the brains of your business unit managers, just waiting to be harvested. You won’t need computational prowess to exploit this crop’s value, just some legwork and—this is key—the most effective presentation tools.

Here we discuss five such metrics, along with some ways to present them visually, as imagined by Andrew Jaquith. Jaquith is a cofounder of the consultancy @stake (which was bought in 2004 by Symantec) and a protégé of infosecurity guru Dan Geer. At @stake he invented a popular analytic methodology that is used to evaluate a client’s risk in its application portfolio. He’s since left Symantec and joined The Yankee Group. More recently he started Securitymetrics.org, a website open to all security professionals for sharing, contributing and advancing the use of metrics in information security. He’s also writing a book, Security Metrics, due out later this year.

Jaquith has sharp, sometimes contrarian opinions on what makes a good metric and what makes for good presentation of metrics. For example, he thinks annual loss expectancy (ALE), a tool used to measure potential losses against probability of losses occurring over time, is useless, because in infosecurity, the L and the E in ALE are wild guesses. Quoting Geer, he says, “The numbers are too poor even to lie with.”

He also thinks CISOs are too apt to dumb down visual representations of metrics for their executive counterparts, mistaking simplicity for clarity. He holds a particular grudge against the overuse of the “red, yellow, green” representation of metrics to signify high, medium and low numbers. “A CEO’s favorite visualization of metrics is a stock chart, a 1-inch square that contains a month’s worth of opening and closing prices, a trend line and several other indicators. Maybe 50 or more data points right there. Don’t tell me they can’t handle complex data. They can, as long as it’s presented well.”

By no means does Jaquith (or CSO for that matter) think these five metrics are the final word on infosecurity. Quite the contrary, they’re a starting point, relatively easy to ascertain and hopefully smart enough to get CISOs thinking about finding other metrics like these, out in the vast fields of data, waiting to be reaped.