CHICAGO -- Penetration testing's future has been caught in heated debate recently, sparked by Fortify Co-Founder and Chief Scientist Brian Chess' prediction that the practice would die off this year. [See: Penetration Testing: Dead in 2009]
Many IT security practitioners rose to pen testing's defense, calling it an indispensible tool for uncovering data breach attempts from inside and outside the organization. [See: 12 Reasons Pen Testing Won't Die]
Move away from the security vendor perspective and one will almost always find that the truth is somewhere in the middle. That's been the experience of Ed Bellis, vice president and chief information security officer for Orbitz. During a presentation at last week's CSO Executive Seminar on Data Loss Prevention, Bellis described pen testing as one of many important tools in his arsenal to protect the sensitive customer data that flows throughout Orbitz's cyber pipeline.
"There are two sides to every story, including the one on pen testing," Bellis said, suggesting that vendors like Fortify will always make sweeping predictions about a technology's future while promoting its own products.
Pen testing has indeed been helpful in detecting weaknesses in Orbitz sprawling network, which includes data centers around the world with thousands of hosts and a cornucopia of internal applications that include an agent desktop, home-grown software to process transactions and back-end security controls. "The number of apps we deal with goes into infinity, and you need a variety of security tools to protect them," he said.
Zeroing in on pen testing, Bellis outlined three specific areas where the craft has proven its worth, and a couple areas where its usefulness is more limited:
Pro: Social Engineering Finder
Social engineering has always been a sure path to a company's sensitive data, and Bellis has found that the weak link is usually an insider who is trying to be helpful with no inkling of the dangers.
"Pen testing will help you catch people who try to use social networking to work their way into a call center," he said. "People working in the call center can be overly helpful when they're trying to help customers, and they can and do get burned in the process."
In this scenario, the pen tester can go hunting for cases where a call center employee is opening the door too wide. Then, those weak links can be addressed, Bellis said.
Pro: Legacy App Finder
As Bellis mentioned, the number of applications in use within Orbitz goes into infinity. Buried among them are apps that have been around forever but may no longer be in use. Yet they are sitting on the network, replete with vulnerabilities waiting to be exploited by a data thief.
In this case, pen testing is helpful.
"Pen testing is a great way to pinpoint legacy apps that are potential trouble -- apps you built years ago that aren't going anywhere," Bellis said. "You'll find apps you didn't know you had."
Some of those applications are easily exploited by company insiders with malicious intentions, including those who have just been laid off. In a separate presentation, Symantec Corp. Data Loss Prevention Senior manager Jenny Yang mentioned a study the company recently conducted with the Ponemon Institute in which 59 percent of those surveyed admitted to stealing confidential company information on the way out the door.
Yang noted that the most common method of data lifting in this case is to put the data on a CD or USB stick. Those methods often involve accessing some of the legacy applications that are a doorway into the more sensitive data stockpiles. "To deal with this, you need to find out where the sensitive data resides, understand how it's used and prevent it from being downloaded," she said.
Pen testing is a useful tool for that task, Bellis said.
Pro: Logic Flaw Finder
Another weak link on a network is a logic flaw -- a vulnerability that can allow someone to access data that appears safe on the surface. Bellis said this is another area where pen testing is useful. "It often takes a person to find a logic flaw [as opposed to automated security tools] and you often find that you don't have to be a hacker to exploit an application in ways not intended," he said.
Example: Many online public relations services like Business Wire store embargoed press releases -- those not meant to be released until a specific date -- on site in an area thought to be closed off from the viewing public. But logic flaws can enable a competitor to access them. In one case, Bellis noted, an Estonian financial firm was able to use a site log-in to stumble upon a competitor's embargoed releases. The firm ultimately made $8 million on insider trading by exploiting this weakness, Bellis said.
Con: Can't See Everything
Among the areas pen testing falls short, Bellis said the craft can't be used to get a panoramic, 360-degree fix on the organizations entire security state.
"You won't find more than 2 percent of all your weaknesses," Bellis said. "You have to prioritize what you want that 2 percent to include, and that can be difficult."
Orbitz's priority is to protect customers from those who would use the company's websites to infect the customer -- a tall order in itself, Bellis said.
Con: Doesn't Always Work
Bellis also noted that like any security tool, sometimes the pen test won't work completely. Sometime a test will fail to find a serious weakness, he said. But then that's why it should only be seen as one tool in a larger security arsenal.
"The key is to know what you're expecting to find with a pen test and set expectations accordingly," he said. "In the end, though, no security tool is 100 percent effective on its own."
Turning back to Chess' prediction that pen testing would die out, Bellis noted that certain security technologies are always being marked for death. There was Gartner's prediction in 2003 that IDS was dead (intrusion detection and prevention systems live on today), for example.
"None of these work completely, but none of these are completely worthless," he said.