Digex CSO Pamela Fusco: The Best Security Defense is Action

Digex prepares for attacks by conducting a full security audit every 24 hours and actively gathers information on exploits and vulnerabilities. The company uses this data to not only identify common attacks but to trace the source of the hostile actions, block them and then try to identify the attackers. Fusco and her group use honeypots, digital forensics tools and an active incident response team, and they share their data with law enforcement where Digex’s clients deem it appropriate.

“Aggressive? Perhaps,” says Fusco, when asked to describe her approach. “Mission-oriented, definitely.” But maintaining that attitude involves a new set of expenses and risks. Jeff Moss, president and CEO of Black Hat, a computer security consultancy and training company, estimates that less than 5 percent of all companies even have an incident response group that can preserve evidence and collect a chain of forensic information to assist in investigating hack attempts. Where Digex’s policies have a search-and- destroy flavor, the prevalent infosecurity approach in today’s corporate world might be better described as hide-and-hope. Fusco admits the aggressive tack isn’t necessarily right for everyone, but CSOs should consider whether, for their particular businesses, the payoff might be worth the extra effort.

Fusco’s team has an obvious motive for pursuing world-class network security. At Digex, the network is the business. The company provides managed-hosting and connectivity services. Its 768 employees and four global operations centers support 509 clients in industries as wide ranging as finance and education. In addition to hosting, Digex also develops and manages enterprise applications, firewalls, e-mail servers and databases. If the network is unreliable or customers find their data is getting poached, Digex won’t hold those clients for long.

It’s a high-pressure environment. A passionate, energetic and no-nonsense woman, Fusco radiates exactly the kind of can-do attitude that nervous clients would want at the helm of the security effort. After working for U.S. Navy intelligence, Fusco was hired by EDS for its government business branch. Fusco says she and her husband, a retired U.S. Marine, are constantly interchanging their security strategies and designs. And Fusco pays attention to details and is irritated by avoidable mistakes.

She notes, for instance, that companies should make sure that their machines are patched and not granting access via vendor default passwords that come packaged with commercial software. “Come on folks!” says Fusco in a tone that she might reserve for an inattentive sailor. “Start with the basics!”

As Fusco recalls, when she arrived at the company in 1998, Digex had to do exactly that. There was no security team, and Digex needed to take baby steps “to get a grip on what we had within our data centers, detail the network and incorporate security at every layer” just to get out of constant fire-fighting mode. But after only a few of months on the job, Fusco says, she got charged up and decided that she wanted to go well beyond the basics and make dramatic improvementsnot only at Digex, but also in the larger security community.

Flash-forward to today: Digex has arrived at a strategy that incorporates aggressive techniques on top of a sound architectural base. She has presented her security strategies to a number of organizations, including the Electronic Crimes Task Force, Internet World and the RSA Security Conference.

For starters, Fusco created the Systems Security Operations (SSO) team by leveraging 300 or so security full-timers. This group’s work is augmented by the company’s internal operations, engineering, product development and client services staff, plus input from clientsin all, more than 1,000 contributors help design and maintain the company’s security plan. Fusco says this extended community helps Digex and the SSO accomplish the security agenda without exhausting their funding and resources. At the same time, with so many chefs stirring the broth, accountability is key. Every security system or process has an owner within Digex who is responsible for auditing, upgrading and redesigning his piece of the plan. “So many organizations procure exceptional security software and hardware, but there is no one allocated to ensure the ongoing capacity and reliability of their security program,” says Fusco.

SSO monitors security data at several levels: system, application and network. Round-the-clock collection and analysis of audit data raises a red flag if servers and peripheral devices on the company’s network are experiencing performance issues or otherwise behaving irregularly. The team then correlates this data with internally developed analysis techniques in a database dubbed SecAudit, which contains definitions of known infosecurity threats and vulnerabilities. The SecAudit database also has a profile of each server within the Digex network and details the type of operating system and applications each is running.

This systems and application-layer monitoring is further matched to information detailing network traffic. Fusco says the company uses digital forensics and real-time network intrusion detection systems, or IDS, at all four of its worldwide data operation centers. Data from these monitoring systems is correlated with internally developed analysis techniques and code that define known exploits and vulnerabilities. Fusco won’t reveal all the details about the specific weapons in Digex’s defense arsenal, but she says the group has customized open-source tools such as Snort, a popular IDS used to identify and curtail network traffic deemed potentially malicious. The Digex group has also written its own custom forensics tools.

At the network and infrastructure level, Fusco says, the Digex security team tracks data on the volume of potential incidents it has thwarted. The group also notes which attempted exploits it has blocked and why, where that network traffic has come from, and what type of traffic it is, using IDSs and data from the network switches. Since the IDSs function in real-time, she says, Digex can compile data for any network entry or exit point at any moment. This approach served the company well when the SQL virus emerged earlier this year. Fusco says her team recognized an increase in attempted attacks and was able to quickly double-check its defenses. Unlike so many companies, Digex says it suffered no damage.

Monitoring isn’t Digex’s only defense, of course. Standard operating procedure includes modifying the default configurations set by the company’s IT vendors to enhance the security of everything from applications to operating systems. The security team has designed “Digex security standard build” configurations for Windows, Solaris and Linux. Digex conducts analysis and testing on software purchased from other companies in support of patch deployments and security fixes, and Fusco says company engineers will often reengineer the products and disable services that violate the company’s security policies. Typical changes include renaming files, renaming or removing certain service accounts, removing services on the system that are not required for that particular system to function, and disabling file transfer protocol, or FTP. At the policy level, Digex pursues certifications and standards such as BS7799.

That’s a solid foundation on which to build, but Digex’s security tactics go further. Since 1999, Digex has also deployed honeypots on its network. Honeypots are lightly defended systems that are set up specifically to allow the owner to spy on network intruders. A hacker who penetrates a honeypot thinks he has compromised the network and may proceed with whatever mischief he intendedbut, in fact, the honeypot will block and record the hacker’s every move, whether he attempts to copy files or create unauthorized user accounts.

Honeypots are less widely used than IDSs and firewalls, and more controversial. Proponents say they offer unique capabilities and can help cull through possible false alerts from IDSs; detractors say honeypots are an unnecessary expense and, if configured carelessly, can actually create a vulnerable point in the network. Fusco herself agrees that a company without a honeypot can often gather the information it needs with a good firewall, IDS, server logs, and vulnerability and assessment scans. According to Fusco, the device should restrict access to other corporate resources and networks. Further detection devices and an alerting mechanism need to be configured in a stealth mode on the honeypot so that the security team can be alerted without tipping off the intruder that he’s wandered into a trap. Ironically, Fusco emphasizes that companies must make sure that patches on honeypots, security platforms and network IDSs are up to date to ensure that these machines are not compromised in unintended ways. “Some people patch all of the other systems and forget to patch the security systems,” says Fusco. (For more on honeypots, see “You Can Catch More Spies with Honey,” available at www.csoonline.com/printlinks.)

Even with all this network armor in place, Fusco’s SSO group maintains a Security Incident Response Team and always generates an “after action analysis” following a security eventeven one that was successfully blocked. “With each new incident or attempted exploit, we improve our security structure,” says Fusco. “No one on this team is on a 9-to-5 mentality.” Daily security reports are e-mailed to Digex executives and key operations personnel, documenting each system’s patch level, services, applications, versions, functionality, address and host names. The daily reports are archived for historical and forensic purposes, and the entire security lifecycle of each server is retained within a restricted database. Fusco says this system lets the company capture the status of each system and provide current, in-depth data to customers and executives.

So why don’t more companies take an aggressive posture toward information security? The biggest obstacle is simply that it’s too costly. Network service providers live moment-by-moment by available, airtight networks. Even in today’s Web-enabled world, relatively few industries have the resources to throw at honeypots, in-house forensics tools and the like.

Black Hat’s Moss says it typically costs about $100,000 to set up and monitor honeypots. He also says dedicated incident response, legal and forensics teams are rare, usually seen only in government organizations and banks, which are required by law to collect this type of information. Moss believes that the only people with any business running honeypots are research institutions, the military or the government; otherwise, he says, honeypots are a waste of time and energy. “You only have so many security dollars. It will get attacked and someone will break in, but [typically] by a script kiddie. Are you going to go after him with a $100,000 investment?” asks Moss. “Why not take that $100,000 and invest it in better people and become more secure with better host-based security, better antivirus software, better firewalls. That gets more bang for the buck.” Honeypots, he notes, can create a variety of problems, including potential confusion for auditors or managers who might mistake them for a vulnerability. Some administrators have been known to call a poorly secured machine a honeypot just to protect themselves. Misconfigured honeypots can also be taken over by intruders and used to harm other computer assets. “The myore simple the honeypot, the more secure it is, and the less risk it is to the system,” agrees another expert, Lance Spitzner, author of Honeypots: Tracking Hackers.

Fusco concedes that measures such as honeypots and digital forensics are not appropriate for every company. Organizations that keep their most sensitive information on back-office networks have different threat models to ponder, she says. In such organizations, it is typically more critical to audit user accounts and resource logs to ensure that employees are abiding by internal security policies. Honeypots, Fusco says, are better for dealing with malicious external threats than internal policy infractions.

Even though Digex’s network security is mission-critical, its defense funds are not unlimited. In fact, as of this writing, Digex’s recent financial performance has been poor, and an acquisition offer from MCI was on the table. Fusco won’t share the hard numbers in her budget and cost-justification process but does say that the company’s leadership agrees it’s money well spent. On the soft side of the benefits equation, Fuscoand her superiorsbelieves proactive information security enhances customers’ trust in Digex. According to Fusco, security issues started to get more board-level attention just prior to Y2K, when companies began to pay closer attention to the security requirements of their enterprise clients. Digex’s customers, especially those in finance, e-business and insurance, began to challenge Digex to adopt a strong security presence, says Fusco. Even so, turning the tide to get buy-in to enforce and implement security within the Digex corporate confines required “a dig in and be persistent strategy,” says Fusco. Metrics have also played a pivotal role. Once basic monitoring efforts were in place, Fusco had ammunition, showing executives samples of daily threats and intrusion attempts. She also gave straightforward calculations of costs incurred and avoided. “You have to provide them with the facts and the aspects and the reality of what is lurking around the corner as a result of a weak security platformno fluff and no exaggerations,” Fusco says.

If more CSOs did so, perhaps aggressive defense wouldn’t be universal, but it also might not be quite so rare.