The great IT risk measurement debate, part 1

IT risk—can it be measured, modeled, mitigated? How much data do we need? Experts Alex Hutton and Douglas Hubbard debate the finer points and reach some surprising and practical conclusions.

By Bill Brenner

February 28, 2011 — CSO —

Risk evaluation models in IT are broken, but we can do more with available data than you might think by correcting for known errors in risk perception. Those are a few of the conclusions Alex Hutton and Doug Hubbard came to in their dissection of risk management. CSO Senior Editor Bill Brenner sat in on the conversation. Here are some highlights.

Update: Also see Part 2 of the discussion (posted 3/2/2011).

Doug Hubbard is the author of The Failure of Risk Management and How to Measure Anything, and is the CEO of Hubbard Decision Research.

Doug Hubbard: Infosec is a very interesting subset of risk assessment and risk management in general. It falls in a category of disciplines that have developed risk management in isolation from what we now know about experimental psychology explanations of risk.

There are a lot of subjective estimates of risk in infosec, and now there's decades of studies about the goofy, quirky things people do that affect our risk perceptions and our risk aversion.

For example, being around smiling people actually makes you more risk tolerant.

Alex Hutton: I would believe that.

Hubbard: Recent bouts of anger or fear change your risk aversion. So does your testosterone level, which changes daily, and maybe whether or not you've had your coffee, or how frustrating your commute was—that changes your perception of risk. So all of these random, irrelevant events have much bigger impacts on our risk assessments and risk management than we probably like to believe.

Read much more CSOonline coverage of critical issues in security metrics

Hutton: [Industry luminary] Dan Geer has said something to the effect that this is one of the most interesting fields to be in in our lifetime. Forgive me if I'm misquoting, but I agree because I see the culmination of risk management done properly as the application of science to the problem. Information risk management—it really is actually relatively unique in terms of security, because the technology changes more rapidly than physical security. Because the threats tend to be adaptive, we don't have data sets yet like you do with car accidents, where year over year, despite changes in technology and so forth, it's a fairly standard number that you can expect. All of the science and all the research has yet to be done.

Hubbard: I get to work in so many different industries on completely different kinds of problems. Right now I'm working on forecasting business opportunities for new pharmaceutical products. Last year I was doing business models for movie industry investments, and in the meantime, I worked on uranium mining, and fairly soon here we'll do risk-return analysis for large airport development projects. All very difficult-to-measure sorts of things. But one of the most common things I hear from all my industry clients is that they're unique among all industries, and I say, "Yes, you are unique and so is everybody else. In fact, you're almost uniformly unique. I can actually name several industries that have these characteristics that you're associating with this."

• Print