Flirting with Disaster: Are You Really Ready for a Crisis?

If a major crisis hit your IT organization tomorrow, would you be prepared? Does your IT department know what to do? Are you coordinated with corporate management? Do you know how to keep key stakeholders apprised of the situation? Have you rehearsed?

Many CIOs have business continuity plans that document procedures for recovering key data and applications in the event of a disruption to their IT infrastructure. What is astonishing, however, is the number that do not. According to the Gartner Group, fewer than 30 percent of Fortune 2000 companies actually have a full business continuity plan in place. A CIO magazine survey shows 44 percent of IT organizations will be designing a disaster recovery plan at some point in the future. This means that almost half of the businesses in the United States are not prioritizing disaster recovery and business continuity.

Even if a business continuity plan is in place, today that is not enough. The types of crises a CIO may have to deal with have moved way beyond the traditional fire or natural disaster that knocks out a data center. The CIO needs to be prepared to deal with crises such as large-scale identity theft, “blow up” of a large, visible project, hacker attacks, the unavailability or defacement of public Web sites, workplace violence in the IT organization, or the sudden appearance of an unknown white powder.

Virtually every organization of any size is dependent on IT systems for basic operations. So, when a crisis hits an IT department, it can affect employees, customers, suppliers, partners, and even the general public. A major IT crisis is no longer a hidden, back office event: it may be very public and have a major impact on the overall image and reputation of the enterprise. As a CIO, the safest assumption is that a crisis is just around the corner. The obvious conclusion is to plan accordingly – in short, to have a crisis management plan.

Anything that can happen, will happen

If terrorist attacks, power blackouts, tornados, hacker attacks, hurricanes, and worms and viruses have taught the world anything, it is that a crisis that affects IT can occur anywhere, anytime – frequently when least expected. And it can happen to any organization, large or small, public or private.

For example, the MSBlaster worm last summer affected as many as 1.4 million computers worldwide, including those at an American nuclear power plant, Air Canada check-in counters, and the U.S. Navy intranet. During the height of worm infection, many users couldn’t even access the Internet. One victim of MSBlaster was the Maryland Department of Motor Vehicles, which was forced to close its doors for two days. That agency was widely criticized for not having a plan in place to continue providing service to citizens seeking to renew their driver’s licenses or registration and for not clearly communicating with the public about the problem, its status, and what was being done to correct the situation.

Business Continuity Planning is a Start

Preparing for a crisis starts with the basics of disaster recovery and business continuity planning. A sound business recovery plan will ensure that the mechanics of data backup and recovery and other steps needed to successfully restore operations after a major disruption are in place.

However, the broad range of events which may constitute a crisis for an IT organization, coupled with dependence that most enterprises have on their IT systems, makes it essential for CIOs to go beyond traditional business continuity planning. The broader domain of crisis management provides very useful insights that CIOs should draw from. Some of these steps include:

• providing the means to backup and restore data needed to support critical business processes;
• providing alternative facilities needed for critical operations;
• providing the ability to operate critical business processes in a degraded mode.

Don’t Leave People in the Dark

Crises impact people in ways that can have lasting negative consequences for an organization. This is as true for an IT crisis, such as a well-publicized looting of sensitive customer data by hackers, as it is for a more general corporate crisis, such as discovering Ricin in an office or a CEO indictment. In the most severe situations, customers, regulators, and investors may lose confidence in an organization.

Even crisis-prepared IT organizations often overlook the need to communicate. Effective response to an IT-related crisis requires understanding the broad set of stakeholders and having a plan for communicating the information needed to keep each stakeholder informed and confident. In particular, in increasingly networked environments where IT systems are closely connected to key customers and suppliers, a crisis effecting one organization’s IT environment can quickly spill over into others. At the very least those with whom we are connected need to be kept informed.

What do people need – beyond basic survival – following a crisis?

• Immediate aid and assurance of safety;
• Information and reassurance;
• Understanding and ongoing support; and
• A rapid return to productivity.

A Clear Plan of Action

To prepare for the possibility of a crisis, CIOs should create a permanent IT crisis management team that includes not only IT experts, but also senior business executives, human resources, legal and public relations professionals who can bring a broad set of perspectives to the crisis at hand. This CIO-led team should be integrated with the enterprise’s overall crisis management team. Depending on the nature of the situation, the IT crisis management team and the enterprise crisis management team may operate as a single entity.

Besides proactively establishing the IT crisis management team, CIOs should prepare and document an official IT Crisis Management Plan. This plan should include:

• The names, roles, and contact information for the crisis management team. One specific role that should be identified is public spokesperson. Consistent messaging, delivered by someone who is used to dealing with the public and the media, is critical in a crisis. Determine how the CIO will assist in providing information for messaging and details needed to complete Q&As.
• A list of internal and external specialists (with current contact information) that may be called on as subject matter experts. This may include security professionals, people familiar with internal applications and data, and vendor contacts.
• Contact information for key government agencies, such as law enforcement, regulatory and public health. What information does the CIO need to provide prior to reporting the crisis?
• Other key stakeholder contacts – these may include senior managers in the enterprise outside of IT, key customers, partners, and suppliers.
• A procedure for communicating with IT staff – including backup if email or the phone systems are not available.
• A procedure for communicating with all employees – including backup if email or the phone systems are not available.
• Other relevant documents, such as the enterprise crisis management plan, government reporting regulations, or corporate policies.
• A designated “communications center” where the crisis management team can meet and have access to internal systems, the Internet, and cell and wired phones.

This document should be distributed to the members of the IT crisis management team, the enterprise crisis management team, and senior IT staff.

Getting Into Crisis Mode

When a real crisis hits, the first step is to notify the crisis management team to gather the facts and assess the situation. The CIO needs to be prepared to provide information that will be used to craft a public statement, such as the following:

• The nature of the crisis;
• Specifics around what the CIO and IT are doing to address the crisis;
• An ongoing assessment of the situation and timeframes;
• Organizations providing assistance.

Providing thorough information is critical to keep rumors and misinformation at bay. Today’s society runs to the television set and the Internet as soon as a crisis hits – so your message needs to be controlled. Additionally, facts should be communicated in one voice, to your constituents. Uninformed and varying messages will hurt the IT organization’s credibility and intensify the crisis. Remember, a CIO is not only dealing with a technology crisis, but the impact of the crisis on the company’s brand, image and customer base.

Practice, Practice, Practice

Obviously CIOs can’t foresee every eventuality or develop a detailed plan for dealing with each potential crisis. What a CIO can do is make sure the staff is accustomed to the crisis management process and is prepared to deal with the unknown. Drills, simulations, and training are essential components to ensure that when a crisis does occur, the right procedures are followed and the situation is dealt with effectively. Drills and simulations also allow CIOs to test the plans and correct weaknesses before they are uncovered by a real crisis. There’s a good reason we see first responders running disaster drills and the military running war games. These exercises provide valuable training and the opportunity to fine tune plans to raise the level of readiness in the event the worst actually occurs.

Sure, there are plenty of problems today to occupy a CIO’s time, but part of being “vigilant and visionary” is to look ahead and to head off potential problems. Having a crisis management plan “in our pocket” and staff that knows what to do in a crisis is some of the best insurance CIOs can have in today’s uncertain world.

Wick Keating is senior vice president and chief technology office of AMS, a global business and IT consulting firm headquartered in Fairfax, Va. For more information, please visit www.ams.com.