How Legacy Systems and Lack of Data Undermine Stress Tests

A recent Fed report noted several issues - including some data and IT deficiencies - that are leading to inaccuracy in banks’ stress tests.

The most recent bout of stress tests revealed a need at several institutions involved in the tests for better data collection and management, replacing aging systems and integrating current systems across organizational silos, according to a new report on the test released yesterday by the Federal Reserve.

The report stated that some banks based their stress test results on assumptions about the bank’s ability to sell off bad assets in a market downturn, or didn’t take factors unique to the institution’s geographic market or balance sheet into account. These assumptions or miscalculations led, in some cases, to weak results in the Fed’s view, and in some cases these mistakes could be traced back to problems with data or IT systems, the report said.

For example, the report noted that several banks’ estimates of their pre-provision net revenue (PPNR) were impaired by data limitations and the use of weak predictive models. “Bank holding companies should not use weak models just for the sake of using a modeled approach to PPNR,” the report said. The banks that used these weak models as a point of reference sometimes then adjusted the results based on expert judgement, but failed to explain and document why those adjustments were accurate, the report said.

Some of the banks were shackled by a lack of data, sometimes working with only 10 previous quarters of data, which the fed claimed was not enough to provide an accurate prediction of how a bank would deal with stressful conditions. These data limitations were sometimes caused by systems mergers or changes, the report explained. Some banks overcame their lack of internal data by leveraging external data, such as aggregated data from across the industry. This could sometimes be more helpful, the report stated, than using an insufficient amount of more relevant or granular data.

In some cases tests were also compromised by challenges with their management information systems, according to the report. The banks best able to ensure the integrity of their results had information management systems that allowed them to collect, analyze and deliver information quickly, and could run analysis to help with their capital planning. But some institutions have systems that are antiquated, siloed or incompatible in some way, and require staff intervention to reconcile across systems.

Earlier this year the Fed announced the results of 18 bank stress tests, with two institutions - BB&T and Ally Financial - failing the tests. BB&T has already submitted its new capital plan to regulators, while Ally Financial has said it will do the same soon, according to media reports. JPMorgan Chase and Goldman Sachs both had to resubmit plans as well. they passed the stress tests but regulators were still worried by some weaknesses in their results.

Jonathan Camhi has been an associate editor with Bank Systems & Technology since 2012. He previously worked as a freelance journalist in New York City covering politics, health and immigration, and has a master's degree from the City University of New York's Graduate School ... View Full Bio