When Disaster Strikes

CU personnel who have experienced disasters offer advice on business continuity planning.

November 12, 2011


When Disaster Strikes


  • Conside employee and member needs as you plan for business interruptions.
  • Test and retest your disaster recovery plan, especially after system upgrades or other changes.
  • Board focus: Regulators expect the board to regularly review and update your credit union’s business continuity plan.


When Louisianans think about floods, they usually think of “water falling from the sky or blown in by the wind,” says Mignhon Tourné, CEO at $300 million asset ASI Federal Credit Union in Harahan, La. As a survivor of hurricanes Katrina in 2005 and Gustav in 2008, the credit union has endured more than its share of natural disasters. But in January 2011, disaster of another sort struck ASI Federal’s corporate headquarters.

During a three-day holiday weekend, a toilet in the credit union’s main branch and corporate offices malfunc-tioned and water poured continuously onto the floor. To make matters worse, rolls of toilet paper stored on the bathroom floor floated over to the floor drain and plugged it up. When some of the credit union’s IT staff stopped by the office later in the weekend, they were greeted by nearly a foot of water on the building’s first floor.

Completely renovating the first floor took three months. “It was a major interruption to our business,” Tourné says. It’s also a reminder that disaster recovery planning must consider not only major weather-related catastrophes, but also less spectacular events.

“As much as people think about hurricanes, tornados, and other disasters, much of what we do relates to events that never make the news,” says Gary Yeager, vice president of sales at Ongoing Operations, a disaster recovery firm and CUNA Strategic Services alliance provider.

Another bit of advice you’ll receive from people who have experienced a disaster is that the recovery process teaches you how to better prepare for future calamities.

Rally and be resilient

After Katrina, ASI Federal set up a permanent disaster recovery site 150 miles away. That location unfortunately was in Hurricane Gustav’s path three years later. “That’s when we started making big changes,” Tourné says.

She ticks off a list of new backup measures:

  • A call center with PSCU Financial Services;
  • Core processing in Troy, Mich.;
  • Servers and data for other critical applications in Dallas.
  • A new disaster recovery site 400 miles north in Memphis, Tenn., where core staff could evacuate to work. The site includes temporary office space and hotel accommodations.

Overflowing toilets, however, can happen anywhere, not just in hurricane country. After the January incident, ASI Federal’s staff pumped out the water and cleaned up so they could return to work on Tuesday (Monday was Martin Luther King Day).

The branch office in another section of the first floor wasn’t flooded. Business opened there as usual on Tuesday. On the corporate office side, staff adjusted. They doubled up in second floor offices and loan officers worked elbow-to-elbow at their computers in the lunchroom. A couple of days later, Agility Recovery—a CUNA Strategic Services alliance provider—brought in temporary buildings and set them up in the parking lot. Lending and human resources staff moved there for three months.

In light of past experiences, “We have an ability to react that might be different from other folks,” Tourne says. “We know how to rally and be resilient as a team.”

Improvise as needed

At $160 million asset Pinnacle Federal Credit Union, Edison, N.J., this summer’s Hurricane Irene taught staff a critical lesson: Always be ready with Plan B. Faced with power outages, “We had to improvise,” says Harry Jacobson, president/CEO.

He’s speaking specifically of what it took to retrieve backup tapes stored in the credit union’s Fairfield branch, 40 miles north of Edison. The branch wasn’t flooded, but it was in the heart of a flood zone, where access was severely limited.

A member of the credit union’s IT staff was sent to Hagerstown, Md., to use Ongoing Operations’ disaster recovery facility. Before he left, he needed the credit union’s backup tapes. But massive traffic jams prevented him from getting to New Jersey for the tapes from his home in Chester, N.Y. “He might have made it eventually,” Jacobson says, “but we needed to move quickly.”

So Jacobson ran an obstacle course to get the backup tapes. Another credit union employee from the Fairfield branch met him in a town they both could reach by navigating around flooded roads and bridges. Then Jacobson drove to Pennsylvania to meet the member of the IT staff, who took the backup tapes with him to the disaster recovery facility in Maryland. After that experience, Pinnacle Federal decided it needs a different backup tape storage site, not yet designated. And “make sure that whoever has to go to the backup site, such as your IT person, knows how to get into the building and where to find the tapes,” Jacobson says. “It could be a building they’re not familiar with. Don’t take anything for granted.”

Next: Take care of employees

Take care of employees

Damon McGregory, facilities manager at $800 million asset Twinstar Credit Union in Lacey, Wash., offers another branch-related tip for disaster planning: “When you have branches spread all around,” he says, “you need somebody on site who can go in, look at equipment, and make a quick evaluation of what’s going on.”

That became apparent when a late 2007 windstorm struck western Washington, toppling trees and power lines. Back then, Twinstar had 20 branches located from the Portland, Ore. area to just south of Seattle. Three branches lost power. The one in Ocean Shores, Wash.—right on the coast about 80 miles from the credit union’s headquarters—was a special concern. “That entire community had no power,” McGregory says, “and it’s more isolated. It looked like that branch could be without power for days.”

Work crews cleared area roads quickly, and branch employees were able to get to work. Agility Recovery brought in generators. “We were one of the few places in town with power,” McGregory says. “We gave the appearance that it was business as usual.”

But life was stressful for employees. “They hadn’t had a hot shower or a hot meal for a long time,” McGregory says. “Sure, they showed up for work. But there was a lot going on in their personal lives due to the storm and power outage.”

That’s why Twinstar plans to bring a motor home or something similar to any future disaster-affected branch. “That way, we can provide creature comforts to employees,” McGregory says. “Next time, we’ll take care of that right away.”

Consider what if…

Tornados wreaked havoc across the south in spring 2011. DCH Credit Union, with assets of $34 million in Tuscaloosa, Ala., was left with holes in its roof and some shattered windows. Two members and all but two employees were in the credit union when the tornado struck on Wednesday, April 27. They locked the doors and headed to interior bathrooms as safe rooms. Everyone emerged without a scratch. “We were lucky,” says Kevin Maguire, CEO. “We didn’t sustain a direct hit. The strip mall right next to us was completely destroyed.”

DCH lost power and Internet access. Agility Recovery delivered generators which, because of the heavy demand for such equipment in the area, had to come from Atlanta, 200 miles away. By Friday, DCH had power, and Agility Recovery brought in satellite equipment over the weekend to provide Internet access. The credit union reopened on Monday.

In recent months, DCH has been revisiting its disaster recovery plan and considering additional disaster scenarios. What if the building had taken a direct hit? What if the computer room had been destroyed? The credit union intends to do more disaster drills with employees, and it also designated the vault as its safe room.

“Before people worried about getting trapped in there if something blocked the door,” Maguire says. “Now, everyone knows about the safety features inside, and security officers told us that the first place they look for people in a destroyed financial institution is the vault. The door will have a flag on it to make sure somebody knows we’re inside.”

Experts also stress the need for testing a disaster recovery plan. Tower Federal Credit Union, assets $2.4 billion in Laurel, Md., takes that to heart. “Our testing is continually evolving,” says Sean Zimmermann, senior vice president of operations and technology. “We test once a year at a minimum, and that’s not enough.” Tower Federal’s testing involves running operations from a disaster recovery site to see if everything works. Tabletop testing—or talking through what-if scenarios and procedures—also enters into planning.

The credit union has weathered two events that had a significant impact on its operations. In 2009, a telephone company construction crew accidentally damaged telecommunications lines into its headquarters. A year later, a hardware malfunction brought down the core system. Both times Tower Federal switched over to its disaster recovery site at Ongoing Operations.

Through these experiences, the credit union learned, “it’s extremely important to have almost all areas of your credit union involved in testing your business continuity plan,” Zimmermann says. Otherwise, he explains, staff might wrongly assume certain functions and information will be available after a disaster.

Also, retest the plan when you make system upgrades or other changes. Testing at that time could eliminate nasty surprises later. “We’ve run into that,” Zimmermann reports. “We think we’ve tested, but when we test again, lo and behold, something doesn’t work because we had made some changes.”

Finally, assess up front the impact a new system or process will have on your disaster recovery plan, Zimmermann advises. “You might be choosing among three new systems, and you have them ranked one, two, and three,” he says. “But one option could cost twice as much for business continuity because you have to replicate it for the disaster recovery site. We consider that at the front end now. It used to be an afterthought, but we’ve moved it up to the initial decision-making stage.”

Next: CUs have compassion


‘CUs have compassion

In April 2011, a tornado leveled vast sections of Tuscaloosa, Ala.

“I grew up in Oklahoma, and I’ve seen a lot of tornado damage,” says Mike Bridges, vice president of marketing and communications for the League of Southeastern Credit Unions. “But I’ve never seen anything like what I saw in Tuscaloosa. It literally was like a bomb had gone off in part of the city.”

Among the destroyed properties were some 100 homes in a Tuscaloosa Housing Authority (THA) development. THA found new housing for displaced residents. “But these people were left without one stick of furniture, one pot, or anything,” says Tommy Cobb, president/CEO of $53 million asset Tuscaloosa Credit Union. THA is in the credit union’s field of membership.

So Cobb joined forces with THA and a local Baptist church to raise money to buy everything from furniture to dishes to toothpaste, and delivered the goods right to people’s doorsteps. About a dozen credit unions from Alabama and the Florida Panhandle contributed $92,000 to make those purchases.

“Credit unions have compassion, and they trusted us to do the right thing with the money,” says Cobb. “It was that cooperative spirit in action,” adds Bridges.













Planning Pointers

A disaster recovery plan shouldn’t “sit on a shelf and gather dust,” says Paul Sullivan, vice president and general manager of Agility Recovery. Experts suggest a few steps for creating a living document:

  • Designate a place to go to restore operations, whether that’s another location or a mobile office brought to your site. “If you don’t have that,” Sullivan says, “your plan is worth no more than the paper it’s written on.”
  • Plan how to communicate with staff. “Your people are your most important asset,” Sullivan says. “Set up a notification system. Reroute telephone calls or establish a special number staff can call for updates.”
  • Focus on more than the core system. “The core is about a third of your IT infrastructure,” says Gary Yeager, vice president of sales for Ongoing Operations. “What about the other two-thirds? If you lose connections to your third-party vendors, you’re in as much trouble as if you lose your core.”
  • Prioritize applications. Figure out your business-critical applications. The core will come first, “but from there, opinions differ on the next critical application, and the next,” says Steve Comer, credit union industry manager for Hyland Software. “Prioritizing your applications is key to how quickly your credit union will get back in business.”
  • Consider geography. Locating a disaster recovery site 10 miles away may seem convenient. But in a natural disaster, “a 10-mile spread won’t help you,” Comer says. “Being in a different state or region of the country is worth thinking about.”
  • Validate backup data. Is the data on your backup tapes any good? “We’ve had clients send us tapes that contain no valid data,” Comer reports. “Too often credit unions overlook the validation step.”
  • Test your plan. Testing also often lands on the back burner, Sullivan says, but it’s the only way to assure your recovery plan will work. He notes that some credit unions test by using disaster simulations. A more basic method is a tabletop exercise in which you “bring in key staff and ask questions,” Sullivan says. “It’s day one, what are you going to do first? What are you going to do next? What will you do on day two? Run those scenarios to see if people are prepared.”

























  • CUNA Strategic Services archived webinars on disaster recovery and business continuity topics: 
  • CUNA Strategic Services alliance providers:
  1. Agility Recovery
  2. Ongoing Operations