In Brief

Building a Disaster Exercise Plan

If things are going perfectly, the people at USAA think they're not pushing hard enough

By Daintry Duffy

November 08, 2002 — CSO — When CIO Steve Yates joined USAA three years ago, the company's business continuity exercises were only on paper. Every year or so, the top-level staffers gathered in a conference room to role-play; they'd spend a day examining different scenarios, talking them out discussing how they thought the procedures should be defined and how they thought people would respond to them.

Live exercises were confined to the company's technology assets where they would conduct periodic data recovery tests of different business unitslike taking a piece of the life insurance department and recovering it from backup data.

In truth, Yates wondered if such passive exercises reflected reality. He also wondered if USAA's employees would really know how to follow such a plan in a real emergency. "Could the company really withstand something massive instead of minor?" he asked. When Sept. 11 came along, he realized the company had to do more. "Sept. 11 forced us to raise the bar on ourselves," says Yates.

So he engaged outside consultants who suggested that the company build a second data center in the area as a backup. After weighing the costs and benefits of such a project, USAA initially concluded that it would be more efficient to rent space on the East Coast. But after the attack on the World Trade Center and Pentagon, when air traffic came to a halt, Yates knew it was foolhardy to have a data center so far away. Ironically, USAA was set to sign the lease contract the week of Sept. 11.

Instead, USAA built a center in Texas, only 200 miles awayclose enough to drive to but on a different power grid and water supply from its San Antonio building. The company has also made plans to deploy critical employees to other office locations around the country.

Yates made site visits to companies such as FedEx, First Union, Merrill Lynch and Wachovia to hear about their approach to contingency planning. USAA also consulted with PR firm Fleishman-Hillard about how USAA, in a crisis situation, could communicate most effectively with its customers and employees.

Finally, Yates decided to put together a series of large-scale business continuity exercises designed to test the performance of individual business units and the company at large in the event of wide-scale business disruption. In March, the company simulated a loss of the primary data center for its federal savings bank unit and recovered the systems, applications and all 19 of the third-party vendor connections. In July and August, it ran similar exercises with other business units.

• Email
• Print
• Comments
• Digg This
• SlashDot

• Disaster Drill: Practice Makes Perfect
• Business Continuity Planning: How Much Is Right for You?
• Business Continuity and Disaster Recovery Planning: The Basics