The Changing Face of Disaster Recovery for Financial Institutions

Wayne Stadnik, VP Continuity Risk Management, TCF National Bank
Wayne Stadnik, VP Continuity Risk Management, TCF National Bank

Wayne Stadnik, VP Continuity Risk Management, TCF National Bank

The examiners are looking for a demonstration of processing capability, meaning that you’re going to have to show that you can actually use the recovered systems to process business transactions.

Disaster recovery for information technology has long been mandated by multiple regulatory agencies across a broad span of industries, but has rarely seen the amount and depth of changes that it is currently experiencing in face of modifications to the regulatory environment. While this change is occurring across many industries, it may be most prevalent in the financial sector.

Afloat in a Sea of Regulatory Requirements

As a result of the financial crisis, the regulatory environment for financial institutions has exploded with a myriad of new disaster recovery and business continuity requirements. For financial institutions the standard has been set by the FFIEC (Federal Financial Institutions Examinations Council) and in February 2015 they released the new Business Continuity Examination Workbook which lays out the new requirements. The FFIEC is not the only regulatory agency that oversees financial institutions; there are many others who have increased their scrutiny including the Federal Reserve Board,

Sarbanes-Oxley, Gramm-Leach-Bliley, the Basel Accord, and if you’re publicly traded let’s not forget the Securities Exchange Commission.

As if the changes to the regulatory environment weren’t enough, as consumers we’ve come to expect things instantly. This means the tolerance for information technology outages continues to shrink to the point where customers expect no interruptions and when they do occur the time for recovery is minimal. This continued demand for always-on operations greatly increases the expectancy for resilient information technology operations. Not to be forgotten, the ever-changing threat landscape continues to provide additional challenges; while you still have to consider the classic threats (power failures, communications interruptions, etc.), there are a whole host of new threats popping up almost daily (consider DD4BC) that can create sleepless nights for any information technology executive. A number of the new regulations include “cyber “response processes. These processes can include anything from an internal cyber response team to utilization of a third party for DDOS mitigation.

All of these changes to the disaster recovery world are creating a tricky path for balancing compliance and operational resiliency. A disciplined approach is required to walk this new road with the focus on reducing risk while improving resiliency.

Testing, Testing, Testing

One of the main things that all regulatory and certification boards are consistent on is the requirement for a robust disaster recovery program. While none of the regulations/standards actually define what constitutes a “test,” all of the examiners will weigh in on the viability of your testing program. They’re looking for a program that is ever-increasing in complexity. No longer is server/system recovery going to be sufficient. The examiners are looking for a demonstration of processing capability, meaning that you’re going to have to show that you can actually use the recovered systems to process business transactions.


Today’s ever-evolving technologies offer a plethora of strategies and solutions to increase the resiliency of your organization whether that’s in your own data centers or in a co-location facility. Virtualization has greatly reduced the need for bare metal recovery when systems fail and when coupled with tools like VMware’s Site Recovery Manager you can achieve a recovery solution that provides automated orchestration and non-disruptive testing recovery plans for virtualized applications.

There are a myriad of data replication solutions that can be used in concert with the virtualization and activation tools to provide an end-to-end solution. Caution must be used as data replication is not the panacea that the vendors claim it to be, as replication is a two-edged sword that cuts both ways. Since data can be replicated in near real-time, this creates the potential for incidents, either intentional or accidental, to be replicated from the production systems to the recovery systems jeopardizing the entire computing platform. Further, consideration must be given to incorporating “air gaps” that create specific points in time into the design of replication processes in order to protect against these types of rolling corruption.

Where Do You Go From Here

In today’s world the examiners are a lot smarter and know what to look for when testing a program, the days of a “checklist audit” are history. Often they’re coming from an operational background which, when coupled with their training requirements, means they’ve developed a much stronger ability to see through any smokescreens.

With all of the regulations and standards out there you need to be careful not to end up chasing the standard-of-the-day. Rather, identify what you are required to be compliant with and align that with a recovery strategy that meets your organization’s risk tolerance and focus your energies there. By taking this approach, you’ll not only help ensure you will pass the examinations, but also put your organization on the path to any certification you may be pursuing.

Read Also


Using "The Box" for Disaster Recovery Planning

Eric J. Satterly, Vice Provost for Information Technology
Disaster Recovery: A Continuous Journey

Disaster Recovery: A Continuous Journey

Mathew Beall, VP of Infrastructure, First American Financial Corporation
Crisis and Incident Management for the 21st Century

Crisis and Incident Management for the 21st Century

Louis Grosskopf, General Manager, Business Continuity Software, Sungard Availability Services