Woodforest National Bank CTO Richard Ferrara: Using Virtualization to Hurricane-Proof

By the time next year's Elite 8 honorees are recognized, Richard Ferrara expects Woodforest National Bank's data center to be 100 percent virtualized. Today, the bank boasts a phenomenal 96 percent virtualization rate as it awaits the imminent release of VMware's (Palo Alto, Calif.) next version -- which promises the additional virtual CPUs and up to 1-terabyte-per-guest support that the bank needs to get over the top, according to Ferrara.

Yet virtualization isn't the goal, says Ferrara, the bank's CTO since late 2008 and SVP since 2006. Rather, it's hurricane mitigation. "Although we started virtualizing in 2005, we really recognized the opportunities virtualization offered with Hurricane Ike in 2008," adds Ferrara, who joined Woodforest, located on the Gulf of Mexico, in 2001 from outside the banking industry.

"I envisioned a new data center consolidation and continuity effort that would allow us to completely fail-over to our colocation at the beginning of every hurricane season in June," continues Ferrara. "Then we'd operate from there until hurricane season ended in mid-October."

It was a gutsy vision. "It's rare for institutions to ever migrate their data centers between locations, much less to do it twice a year," says Ferrara, who leads a 160-person IT department with a $45 million annual budget.

Getting There From Here

To accomplish the goal, Ferrara reports, Woodforest ($3.3 billion in total assets) started in 2009 with a refresh of its Hitachi Data Systems (Santa Clara, Calif.) SAN, and in 2010 the bank began the necessary upgrades to both the primary data center in its The Woodlands, Texas, headquarters and the colocation facility in Austin, Texas. Woodforest upgraded to new HP (Palo Alto, Calif.) blade servers, reducing 10 server enclosures in each data center to five -- one of which in each location stands empty for growth. "We started with the primary data center and moved virtualization up from about 40 percent to about 80 percent," Ferrara comments.

"We also reduced a switch layer and all the associated cabling," he continues. "We replaced bundles and bundles of copper with a couple of strands of fiber. In turn, this significantly decreased the number of ports required in the remaining switches, which makes upgrades less expensive."

Then Ferrara's team took virtualization to the next level. "We were frustrated with the speed of our core systems, which were housed on IBM (Armonk, N.Y.) boxes running AIX [a series of Unix operating systems]," says Ferrara, who combines an easy-going, collaboration-centic demeanor with an intensity for excellence. During a strategy session, he recalls, one team member suggested using RedHat (Raleigh, N.C.) Linux instead.

"So we decided to conduct a skunk works project," Ferrara adds. "Our core systems ran faster on Linux than AIX -- even prior to optimization. This is how I work with my group -- I treat my team more like peers, where I sit down and talk with them about ideas, because I don't have all of the answers."

Moving core systems to Linux, Ferrara says, pushed the bank's virtualization to where it stands today, and in June 2011 Woodforest conducted the first total fail-over to the colocation facility. What's more, according to Ferrara, the entire fail-over process -- which takes just four to eight hours over a weekend -- impacted customers for less than an hour. "And we plan to eliminate the customer impact entirely by eventually having the data centers run live-live," he notes. "Overall, the goal of our technology department is that the customer never sees the impact of our work, even when we're migrating from one data center to another."

Indeed, Woodforest's IT architecture enables near-complete transparency for all types of administrative chores, without impacting customer service, Ferrara says. And that's saying something for an institution that operates 750 locations in 17 states and provides access to live representatives 24/7.

"Our core sits on the back end and, every morning, all of the information downloads into our [Microsoft] SQL middle tier," Ferrara explains. "The middle tier is connected to all of our customer touch points, which permits us to work on most systems at any time without affecting our customers."

But that's not good enough for Ferrara, who relentlessly automates tasks to push technology further into the background. One example was dumping a brand-name replication tool. "It didn't allow us to take full advantage of the flexibility and mobility of our virtual infrastructure," he says. "At the end of this hurricane season, we're planning to use [Herzelia, Israel-based] Zerto for VMware replication to migrate back to our main data center, which should make migrations even faster."

Humble Beginnings

Fittingly, Ferrara's affinity for exploring emerging technologies helped launch his career in IT. It all started in the 1980s with a spreadsheet, he recalls. While Ferrara was working in Texaco's New Orleans accounting department, his future wife, Donna, introduced him to Lotus 1-2-3 and the power of macros.

The encounter, Ferrara notes, led him to a professional "a-ha" moment. "I realized my department's 'dumb' mainframe terminals could be replaced with PCs, which were a completely new technology at the time," he relates. Ferrara says he foresaw employees running terminal-based applications and utilizing early versions of now-ubiquitous office productivity software. Not long after proposing his idea to a friend in Texaco's IT department, Ferrara left accounting, and the rest is history.

Ironically, Ferrara now has come full circle, as one of his Woodforest initiatives is implementing a virtual desktop infrastructure (VDI) for critical back-office functions, such as item processing. Of course, this permits working from "dumb" workstations -- located anywhere -- connected to virtual desktops in Woodforest's data center.

Ferrara already offers a few VDI lessons learned: "Start with your most challenging group with respect to applications," he advises. "If you can win them over and solve their problems, the rest of your rollout will likely go more smoothly. And begin with a few select members of that group to help you work out all of the issues before moving to the group as a whole."

Going forward, Ferrara foresees BYOD, or "bring your own device," as the next big thing in banking. "The younger generations, especially, already own all of this cool technology, so we need to investigate how we can take advantage of it," he asserts. "It may not work in the branch world. But in the back office, people might not even have their own desktops because they can connect via a VDI session. I see that as the future."

Regardless, it's a sure bet you'll find Ferrara's fingerprints side by side with his team's as they work to leverage new opportunities. "I don't think of myself as purely a manager," observes Ferrara, who once upgraded an employer's Moscow office by setting up the servers in his hotel room. "I enjoy brainstorming and assisting with architecture and database design," adds the avid gamer, renowned for brutally crushing all help desk gamers during his first days at Woodforest.

"Staying hands-on helps me relate to my team members better because I understand their roadblocks," he adds. "I also think it helps us innovate and makes our workplace more fun. If you're not having fun, it just makes for a long day."

Anne Rawland Gabriel is a technology writer and marketing communications consultant based in the Minneapolis/St. Paul metro area. Among other projects, she's a regular contributor to UBM Tech's Bank Systems & Technology, Insurance & Technology and Wall Street & Technology ... View Full Bio