Don't Press the Panic Button

You've just been hacked. Now what? Here's how to avoid resorting to panic mode. At 12:12am on May 1, 2001, in the wake of the tense standoff over the downed US spy plane on China's Hainan Island,'s intranet underwent a drastic redesign. A black backdrop replaced the usually staid blue log-in page, and across the page a stripper danced to music, gyrating her way in (and out) of a purple bikini. Vivid red text adorning the top of the page conveyed an almost cartoon-like threat: "We'll get back at you America!"

Upon discovering the site's saucy new mascot the next morning, nobody knew quite what to make of it. "I thought this was some sort of joke," recalls Ed Prado, president and CEO of the California-based company. "Internally, there was absolute disbelief over what had happened."

The first reaction to a security breach is almost always denial. This must be a network glitch or a stupid joke. Once the severity of the situation sinks in, however, a variety of emotions ensue - anger at the perpetrator, betrayal by the security vendors that didn't prevent it from happening and finally, sheer panic. "Anarchy looks organised compared to [the first 24 hours after an attack]," says Mark Rasch, former head of the computer crime unit at the US Department of Justice and now vice president for cyberlaw at New York City-based Predictive Systems, a security consultancy. By the time companies have worked through these emotions and started addressing the problem, they've wasted precious time - and the situation has likely worsened.

The reason for all the panic is that many companies don't have a well-defined incident response plan to guide them when a virus or a hacker fells their systems. They don't know who to call for help, when and how to communicate the problem to their employees, customers and the media, or how best to get back online.

But companies can no longer afford to make these decisions on the fly because the cost of security breaches is increasing exponentially. The 2001 annual computer crime survey conducted by the Computer Security Institute (CSI) and the FBI reveals a disturbing upward trend in the cost of breaches, and it suggests that a company's chances of facing a security threat are quite high. Of 538 US corporations, government organisations and universities that responded to the 2001 survey, 85 per cent admitted that their security had been breached in the last 12 months. Of those, 35 per cent were willing and able to quantify their losses. Those 186 organisations reported a whopping combined total of $US378 million in financial losses. In contrast, losses from 249 respondents in 2000 totalled a distinctly smaller $US266 million.

By those standards, was relatively lucky. No customers viewed the dancing stripper, and the damage was limited to a few hours of lost productivity as employees processed customer orders manually while the site was down for repairs. But you can't depend on luck. To minimise the damage of security incidents, your company should make some basic choices about how you will approach potential security situations, codify those choices in a detailed, written incident plan that can serve as a blueprint in the event of a crisis, and decide ahead of time who will be in charge of implementing that plan (see "Have a Plan,"). As you develop your incident response plan, you need to think through the following critical issues. By preparing ahead for a possible breach, you can avoid resorting to panic mode and make the recovery process faster and smoother.

Page Break

Avoid the Quick-Fix Trap

When confronted with a Web site defacement, the first impulse is often to take down the site, fix it quickly, put it back up and hope that nobody noticed. But a rushed fix can make matters worse. The security holes that allowed hackers to change the page in the first place remain open for someone else to find and exploit; since hackers frequently brag about their successful ploys on Internet message boards, companies are almost guaranteed an endless stream of re-hacks. Many hackers also build a back door into their handiwork, allowing them to easily get back in and do more damage later. Implementing a quick fix can also quash a company's ability to track down and prosecute the perpetrator. In their haste to restore their sites, companies trample and sometimes entirely erase the crime scene. So the first step is assessing the extent of the damage. Was the hacker just creating a little harmless graffiti or did he gain access to critical customer or proprietary information?

Often companies will find that their initial reading of the situation was completely wrong. On one of Rasch's assignments, a company initially thought an employee was trying to launch a denial-of-service attack (the act of incapacitating a network with a flood of traffic) using some Eastern European computers. Upon investigation Rasch and his team discovered that the employee in question had downloaded some apparently harmless gaming software that was trying - unbeknownst to the hapless gentleman - to connect him to a porn site. But because the software had the wrong Web site address, it kept trying different addresses, thus making it look like the employee was launching a series of attacks.

Page Break

Watch Your Backup

One technique for buying time to investigate without jeopardising the business is to maintain backup servers with frequently updated copies of all Web site pages. A company hit by a security breach can then run its site from the backup servers while combing through evidence on the primary system. The cost of this proposition varies widely, depending on the size and dynamics of the site in question. As a ballpark estimate, the one-time licence fee could run in the thousands, and ongoing annual maintenance costs likely in the hundreds. But having backup for a larger site may be worth it when you consider the value it offers. A company can immediately bring a clean copy of the site back up, examine the damaged site to determine in detail what happened and avoid a rushed fix. When San Francisco TV station KPIX had its online news Web site hacked last year, Webmaster John LeBlanc learned an important lesson about having backup systems at the ready. In the first hack, someone got into the site and posted fictional news stories. "They were all pretty far-fetched," LeBlanc notes, so visitors would likely have recognised the gag. The station fixed the first hack and patched the holes. But in a second incident, someone broke in to one of the site's main servers and changed the root user password - meaning that nobody could log on to the Web site.

Those two incidents prompted KPIX to change its approach to security. "We've set it up so that we can dynamically reroute requests to other servers and can bring up another machine temporarily," LeBlanc says. He also emphasises the importance of frequently backing up all systems so that recent copies of Web pages and critical data are always available if the originals are damaged or altered.

Page Break

Call In Your Team

In almost every case, calling in the incident response team (see "CyberCIRTainty", CIO March) should be at the top of the list of action items on a company's incident response plan. This cross-functional group of executives and representatives from IS, each business unit, public relations, the legal department, marketing and communications, and human resources must be in place ahead of time and trained how to respond in the event of a security breach. "When I see that, I have a big smile on my face," says Bob Weaver, assistant special agent in charge of the Secret Service's New York City-based electronic-crimes task force, "because I know that company is prepared to protect intellectual property and operational data. That's a good business model." The company needs to have at least three ways of contacting each person on the team, because attacks frequently occur during off-hours and knock out forms of communication such as e-mail. "I know exactly how to contact everyone right up to the CEO, 24/7," says the security director for a US-based Fortune 100 technology company that suffered through the I Love You virus last year. (The company deals with an average of 1600 hacking attempts per month - 150 of which it classifies as serious.) "If I was notified that our site was defaced, I could let [the CEO] know straightaway, because he can very quickly engage the business units internally," says the security director, who we'll call Peter Baker.

Corporate politics can hinder the effective execution of an incident response plan. So Baker strongly advises companies to endow the incident response team leader with the ability to overrule department heads and executives. He points to cases within his company when business unit heads were approached with evidence of possible internal security breaches. They viewed investigations as a waste of time and tried to stop them. With the CEO's backing, the security director was able to override the business unit heads, and in at least one case, the investigation turned up incriminating information that resulted in the termination of some employees.

Page Break

Know What To Say and When

The reflexive response during a security breach is often to shut down the lines of communication for fear that the news will leak to the media. But proactive communication can be a powerful weapon during such an event. The company's incident response plan should detail how and when the situation will be communicated to the company's employees, customers and business partners, as well as to the media so that during a security emergency, executives don't waste time hemming and hawing over whether to go public. In the case of employees, the message cannot get out soon enough. Baker's Fortune 100 company navigated the I Love You virus with far less technical disruption than many of its counterparts, thanks to its effective employee communication strategy. When the virus hit, the company invoked its incident response communications plan to ensure that each of its more than 20,000 worldwide employees understood the situation and their responsibilities. On the first morning of the attack, as employees made their way to their desks, they walked by signs and were handed fliers alerting them to the virus and detailing what they should look for in suspicious e-mails. In addition, each employee got a voice mail and an e-mail message repeating the information. "On the first day we had these mechanisms in place, and by the third day everyone was very suspicious of different-looking e-mails," Baker says.

The plan was a technical success as well. While other companies had to shut down their e-mail servers for up to four days to cope with the attack, Baker's company had to pull the plug only on its European e-mail servers and just for a single two-hour period.

When it comes to telling the outside world about a security incident, communication is easier when you're fighting off a well-known, widespread virus. It's harder to decide what to say and to whom when only your company has been hit. That is why it is important to engage PR people who are well trained in technology issues and have a deep understanding of the sensitive nature of security issues. By being proactive, they can help a company turn even a potentially reputation-damaging situation into a positive story.

"Be prepared to come clean before the press does," advises Pat Donnelly, a director in Aon Corporation's Technology and Telecommunications Risk Group in Chicago. When companies come out with the news of a security breach first, they can portray themselves as good corporate citizens rather than let the media paint them as victims or villains. In certain cases companies have a legal obligation to come forward. For example, financial services companies must file a suspicious activity report with the Securities and Exchange Commission (SEC). And in any industry, companies can be held legally liable for failing to warn a critical business partner that its systems have been placed in jeopardy.

Page Break

Going Public

Deciding whether to alert the authorities has always been a sticky issue for corporations that often equate calling in the police or special task force to going public. But the high cost of an attack has led an increasing number of companies to eschew the "don't tell" mandate of years past; they're not only getting mad about security attacks, they're trying to get even. A company's incident response plan should detail whether the authorities will be called in each given case. For example, when an employee receives a threat via e-mail or trade secrets have been compromised, calling the cops is the obvious choice. However, if an employee is suspected of accessing information that's considered off-limits, it could be a matter best dealt with in-house. The security team members at Baker's Fortune 100 company have purposely formed strong relationships with members of the FBI and the Secret Service so that if attacked, they're not just calling an anonymous toll-free number. The company's policy is to call the authorities in the case of a quantifiable financial loss, theft of data or business disruption. But the attacked company must prove a financial loss; according to Baker, US federal law enforcement won't get involved if the financial loss is less than $US50,000.

1 2 Page 1
Page 1 of 2
The 10 most powerful cybersecurity companies