How to Conduct a Vulnerability Assessment

Roger Johnston talks about how aliens, Elvis impersonators and your worst security users can help you find and fix security problems.

CSO: Vulnerability disclosure has been especially contentious in the field of IT security. (See "The Chilling Effect" from the January 2007 CSO.) Does this Vulnerability Disclosure Index apply to IT vulnerabilities as well?

ROGER JOHNSTON OF THE LOS ALAMOS NATIONAL LABORATORY'S VULNERABILITY ASSESSMENT TEAM: It's really meant for physical security. IT lives in a very different world. Let's say you're playing around on your home computer, and you find a very serious software vulnerability. There's some controversy, but most people agree you should do the following: You should contact the software company and say, "I think there's a problem here." You give them a chance to fix that. If after a while they're just stonewalling and not doing anything, then maybe you go public. Once they fix the problem, it's no big deal. Everybody who bought the product typically does checks on whether there are upgrades.

Physical security is not like that. In many cases the physical security systems are from a bunch of different vendors and may be put together by a third-party vendor. Often there's no one company to go to complain about a potential vulnerability. Moreover, the fix isn't just some software download. The fix may require servicepeople going out and changing parts, and it could be very expensive, very disruptive. Before you get everybody all wound up about a physical security vulnerability, you may want to think about, is it even going to be practical to fix it?

CSO: You've written that when the vulnerability assessment is chartered, the sponsor owns the findings, but that that doesn't necessarily "relieve the vulnerability assessors of their responsibility to warn others of a clear and present danger." This might strike fear into the hearts of CSOs who think they're going to hire someone to do a vulnerability assessment and the contract will ensure that the findings remain private.

JOHNSTON: A typical example would be if a company is considering a commercial security device. Let's say we do a vulnerability assessment on that device and oh my gosh, if you poke it with a paperclip it will quit working. And we know that commercial device is being used for a wide variety of applications, including corporate security, U.S. national security and nuclear safeguards. We believe we have some moral responsibility to tell people there might be a problem. Most companies we've done that for have had no problems, and in some cases encourage us to do exactly that. ##