How to Conduct a Vulnerability Assessment

Roger Johnston talks about how aliens, Elvis impersonators and your worst security users can help you find and fix security problems.

By Sarah D. Scalet

Page 3

CSO: When you actually do the assessment, are there warm-ups you can do to get yourself in the mind-set of a bad guy, or are there ways you should set up the room?

ROGER JOHNSTON OF THE LOS ALAMOS NATIONAL LABORATORY'S VULNERABILITY ASSESSMENT TEAM: A lot of vulnerability assessment needs to be very similar to classic brainstorming. A lot of the tools that are applied to creative thinking in other fields can be applied directly to vulnerability assessments. This is kind of a radical position. A lot of people in the security business are not comfortable with this 1960s hippy, touchy-feely, "let's all get together" approach.

CSO: I'm imagining a bunch of beanbag chairs.

JOHNSTON: Yeah. A lot of people would much rather have a rigorous, quantitative approach, and I would claim that's largely a sham. I don't think it's a mistake to use analytical tools like a security survey, but we would like to combine those more closed-ended, straightforward tools with creative thinking. The fact is that creativity has been studied extensively over the last 50 years, and there's a lot of understanding of how you create an environment where people come up with good ideas. It's not quite the seat-of-the-pants, wacky kind of thing that it might look like from the outside.

CSO: Should the CSO even be there?

JOHNSTON: You don't want the boss in the room, because it constrains people. What you need are really nutty ideas, so we strongly encourage thinking about attacks that involve Elvis impersonators and flying monkeys and the use of space aliens. Early on, it's very important not to editorialize. Later on, we're going to prioritize them and think about the practicality of them. In many cases, we have people say, "Well, if I had the space aliens come down with a ray beam, they could do the following." Later on, it turns into a very viable attack, once we get rid of the space aliens and the laser beams.

CSO: Does this take hours? Days? Weeks?

JOHNSTON: It depends. If you're looking at a very complex security program, you may want to spend two or three weeks just kind of freewheeling. But you don't just sit around and do ideas. You generate nutty ideas, and then you go back to the program or the hardware and play around a little bit to see if those nutty ideas might have some merit. Then you get back together again, and you think of more nutty ideas based on what you learned. We're very much in favor of hands-on work, and not just thinking in abstractions. Toss the device around. Chat up the security guards. Kick the fence. Play with the system and try to understand how it behaves.