Incident response plans badly lacking, experts say

There were plenty of security lessons to be learned from the recent Gawker Media breach. One of the lessons that has been glossed over was the failure of Gawker to have a plan in place to deal with a serious security breach, as the company’s chief technology officer Tom Plunkett admitted in his now famous memo:

“First, we never planned for such an event, and therefore had no systems, or processes in place to adequately respond. Our focus as a team (and company) has been on moving forward. This put up blinders on several fronts. As a result, numerous wrong decisions were made by me this past weekend in responding to the security breach.”

When a breach of personally identifiable information goes public, whether it be financial, private health care data, or several million usernames and passwords dumped on the Internet: it’s usually easy to tell which companies had a plan in place, and which organizations did not. Generally, those breaches where an organization has a security incident response plan in place unravel (publicly and internally) in a manageable and coherent way A breach is identified, investigated, and notifications and remediation services (if relevant) are sent to all those affected.

That is how it precisely does not go for organizations without a plan: news stories where conflicting information is published, and it quickly becomes clear that the business does not have a handle on the extent of the breach. Everyone starts to panic: the breached organization, its partners, and the affected customers. If the situation is bad enough even law enforcement and regulators will get vocal. Competitors start to salivate.

Also see: Incident Detection, Response, and Forensics: The Basics

“It doesn’t take long for these situations to fly quickly out of control when companies don’t have an incident response plan in place,” says Brian Honan, founder of Dublin, Ireland-based information security consultancy BH Consulting and Founder and lead of Ireland’s first Computer Emergency Response Team.

If Daniel Kennedy, partner at managed security services provider Praetorian Security Group LLC is correct, more companies than not are flying without any plan at all. “Most firms, even large firms, aren’t far long with their security incident response plans,” he says. “Some large firms have a plan, but it’s filed with the business continuity plan and rarely looked at.”

Straight lines of communication

In the vast majority of the time, companies learn of breaches from partners, customers and others with which they do business. Those partners, explains David Mortman, contributing analyst at security research firm Securosis, will call whomever the contact is that they have on file. That could be anyone from a high ranking executive to a product management or clerk somewhere within the organization. “Once that happens, these things go sideways quickly,” Mortman says.

In these situations “sideways” has serious consequences: evidence is lost, the press is notified before the company knows the true scope of the breach, bad information is disseminated and other crucial mistakes are made.

All of which, experts agree, could be largely avoided by putting a plan in place — and having it followed.

“Breach responses always go right because the business is prepared; they have the right people and processes in place. People aren’t running around wondering what are they are going to do next,” says Honan.

Keep the initial breach incident team “tight”

Kennedy recalls an example when a breach notification almost went badly, but the situation was saved by a manager who took control. “A vendor of the company called a product manager and said they were providing a breach notification as required by state law,” he says. That product manager then called the product developer and requested that a troubling link to the application be simply be removed to fix it. “The developer didn’t think that sounded like the best, or complete coarse of action, and the developer made the decision to call IT security,” he says.

From there, the IT security team took the right steps and fixed the application vulnerabilities and investigated the extent of the breach. “From the point that IT security got involved and tool control, everything worked,” says Kennedy. “Companies that respond to incidents correctly have their response teams well defined and trained,” adds Mortman.

The initial security incident response team should be small, most advise. It should include IT management, IT security, and legal until the scope of the breach is understood. Once the nature of the breach is understood, it’s then often the right time to inform business management and other stakeholders — before making the public announcement. It’s also crucial to establish the right external relationships ahead of time, Honan advises. “Good external links with partners such as ISPs, telecommunication providers and law enforcement are important. Just knowing who to talk to in your local law enforcement office for cyber crimes can help you get over a lot of hurdles,” he says.

Finally, all the experts agree, is to not just put a plan on paper and forget about it. Make sure all employees know who they should contact for anything involving a potential security breach or incident. Then test the plan: contrive a hypothetical incident and make sure participants react the way they should. The test needs to be realistic. “Make it so that one or more of the stakeholders are sick, or the CISO is unable to be reached, and determine how the team responds,” Mortman says.

That advice is as good as it is important, because in the event your business does suffer a breach, you don’t want to give customers, regulators, partners, the media, and even competitors anything to gawk at.