What would you conclude if we told you that large bank failures generally only happen during severe economic recessions, but small bank failures happen even during modest economic downturns? In Lorenc and Zhang (2020) “How bank size relates to the impact of bank stress on the real economy,” two researchers reported just this result and offered it as evidence that the failure of a large bank worsens the severity of a downturn and therefore is much worse for the economy than the failure of a small bank. Others have taken this finding and cited it as evidence that mergers that create banks over a certain size threshold should not be permitted. While the empirical observation reported in the paper is no doubt correct, there is another explanation, of course: large banks are more resilient than small banks, so it takes a bigger economic downturn to cause them to fail. (In economist talk, when two variables each simultaneously determine the other, a simple correlation between those variables does not measure the influence of one on the other in just one direction.) Below we show how the results reported by Lorenc and Zhang’s paper could be explained entirely by large banks being more resilient than small banks.
The Methodology of The Paper
“How bank size relates…” looks back at how GDP growth differs across quarterly periods when banks of different sizes failed. For each quarter going back 60 years, it measures the total assets of banks that failed during the quarter, but only includes banks exceeding some minimum size threshold. It repeats this measurement multiple times for varying minimum size cutoffs. For example, one measurement only counts failures of banks that were in the top 0.5 percent of all banks by size in that quarter. Another measurement counts failures of banks that were in the top 0.4 percent. And so on. It then examines the relationship between GDP growth in each quarter and each bank failure measure in that quarter.
The authors find that an increase of failed bank assets was associated with lower GDP growth, and that the negative relationship steepened as the size cutoff went up. For example, they find that a 10 percent increase in the assets of failed banks in the top 0.2 percent of banks ranked by size was associated with GDP growth that was 0.6 percentage points lower, and that a 10 percent increase in assets of failed banks in the top 0.9 percent of banks was associated with GDP growth that was 0.2 percentage points lower.
They then conclude that:
…the failure of a single bank—above the 99.8th size percentile and with an assumed $200 billion in assets—would result in approximately a 150% decline in quarterly real GDP growth, while failures of five banks—each above the 99.1th size percentile and with an assumed $40 billion in assets—would result in approximately a 33% decline in quarterly real GDP growth. (p.9)[1]
What Is Wrong With the Analysis in “How Bank Size Relates to the Impact of Bank Stress on the Real Economy”?
We have many concerns with the Lorenc and Zhang analysis, several of which can be found in Covas (2018), published soon after the working paper version of the paper was released. However, in this note, we highlight one fundamental problem. Yes, economic growth can be affected by bank failures, but the likelihood and severity of bank failures also are clearly affected by the economy. You can’t analyze two simultaneously determined variables the way Lorenc and Zhang did and expect to disentangle and thereby understand cause and effect.
Suppose, for example, you measured the average crowd noise each time Juan Soto was at bat in Nationals Park and learned that a 1-decibel increase in volume was associated with a 1 percent greater likelihood that he hits a home run (average crowd volume from the moment he steps into the box to the moment he leaves the box). Yes, partly, you’d be identifying how his batting improves in response to cheers, but you’d also be learning how the crowd reacts when he hits a home run.
Economists would typically solve this problem by finding a third variable, an “instrument,” that causes bank failures but is not related to the state of the economy – insider dealing or defalcation, for example. One then would examine how a predicted rise in failures caused by that third variable in turn influenced GDP growth. A well-chosen instrument allows the experimenter to “identify” the causal relationship in just one direction. The authors, however, did not find and use such an instrument.
Lorenc and Zhang may have judged their statistical approach to be legitimate because they based it on a 1983 paper by Ben Bernanke. Specifically, Bernanke looked at how industrial production and bank failures moved together during the Depression and concluded that bank failures reduce economic activity. However, 35 years later no less an authority than Ben Bernanke recognized that this statistical approach had problems. In Bernanke (2018) he stated “However, my empirical work on the period relied heavily on aggregate time series, making it subject to the usual concerns about endogeneity and identification.” “Endogeneity and identification” is economist-speak for “you can’t measure the correlation of two variables that each influence the other and think you have identified an underlying relationship going just one way.”
Bernanke discusses the fairly extensive set of subsequent papers that use a variety of innovative techniques to reduce the problem, including in some cases by finding clever instruments. A few examples of the authors’ descriptions of the challenges with their analyses that they overcome are illustrative:
… Although the bank distress variables capture disruptions in the supply of credit, they might also reflect local economic weakness.
Carlson and Rose (2015)
… estimates of the role of financial frictions identified solely from variation in local bank failures, common in the existing literature on the Great Depression, may instead reflect shocks to local economic conditions that simultaneously affect bank health and firm investment opportunities.
Benmelech et al (2017)
Similarly, economic conditions are likely to be really bad if a large (and well diversified) bank fails. It may then not be surprising if the relative size of the failed bank in the county appears to be negatively correlated with subsequent recovery rates.
Ramcharan and Rajan (2017)
The final quote in particular points exactly to our concern about the most recent analysis.
An Alternative Explanation
To illustrate the statistical problem, we generate our own data and then use it to conduct an analysis that is identical to the analysis in “How bank size relates to the impact of bank stress on the real economy.” We label the data “GDP” and “bank failures” but because we are manufacturing the data it could just as meaningfully be labeled “agriculture production” and “farm closings” in Middle Earth.
Specifically,
- We create a random variable that we equate with economic stress.
- We tie GDP growth to the stress variable and bank failures to GDP.
- We define three size categories of bank failures (small, medium and large). Bank failures in each category only occur if GDP growth falls below a specified level. The threshold decreases (becomes more negative) as bank size increases. If the threshold is passed, bank failures in that category are defined to be a positive random number.
We then examine the relationship between GDP growth and bank failures. We find:
- When we group together medium and large bank failures, a 1 percent increase in failures results in a 0.29 percentage point decline in GDP growth.
- When we remove “medium,” thus effectively increasing the size cutoff, a 1 percent increase in failures results in a 0.44 percentage point decline in GDP growth.
That is, even though in this perfectly controlled experiment bank failures have no effect on GDP growth by construction, we find not only that bank failures reduce GDP growth, but also that the failure of a larger bank does so by more than the failure of a smaller bank. Because large banks are more resilient to economic stress in the manufactured data, large bank failures appear to “cause” bad economic outcomes. A spreadsheet containing the analysis is available (here).
Don’t the “Robustness Checks” in the Paper Show There Isn’t a Problem With Endogeneity and Identification?
The paper includes several different variants of their analysis to evaluate its robustness, but none of them resolves the problem we have identified. In each case, the results can still be explained entirely by large banks simply being more resilient than small banks.
For example, the paper uses regional data in addition to national data and claims that this version of the analysis should address simultaneity:
To the extent an omitted variable might be correlated with both macroeconomic performance and bank failures, it is unlikely that an omitted variable would have the same effect on all regions of the country simultaneously. (p. 9)
However, the same problem exists within regions. If a regional economy is doing poorly, bank failures in that region will rise.
The paper also reports a statistical test used by economists to test the ability of one variable to forecast another (a Granger causality test). The test consists of examining whether past values of one variable can predict the current value of another variable and vice versa. The results are mixed. They find that for banks below a certain size threshold, bad economic outcomes predict future bank failures but above the threshold bank failures predict bad economic outcomes. This pair of findings, however, does not imply a causal link (both the failure and the economic deterioration may be caused by a common, underlying source of stress); nor can it provide any quantification of the relationship between bank size and systemic effect.
Are Large Banks More Resilient Than Small Banks?
There is good reason to conclude that the better explanation for Lorenc and Zhang’s results is in part that large banks are more resilient to economic shocks than small banks. Other economic analyses find evidence of such resilience. Hughes, Mester, and Moon (2001) find that large banks achieve a better risk-profit tradeoff than smaller banks because diversification reduces their risk. Gotz, Laevan, and Levin (2016) find that a one-standard-deviation increase in bank diversification across metropolitan statistical areas reduces bank risk by one-fourth.
In a speech in 2018, then-President of the ECB Mario Draghi, when describing the benefits of European integration, said
In the United States for example, retail banking integration has led to a significant increase in the number of multi-state banks. That was not always the case. For example, following the oil price collapse in the mid-1980s, almost every bank in Texas failed, creating a state-wide credit crunch. One reason was that banks were not allowed to operate across states, so the balance sheets of local banks were completely concentrated on their home state.
In a more integrated US banking sector, banks have geographically more diversified loan-books and deposit bases. By offsetting losses made in crisis-hit states with gains in other states, US banks are more resilient to local shocks and can keep their lending stable.
Conclusion
The estimates presented in “How bank size relates to the impact of bank stress on the real economy” do not shed light on the fundamental relationships between the size of a failed bank and the economic impact of its failure. In particular, the paper’s conclusion that the failure of a single $200 billion bank causes three times the economic damage of the simultaneous failure of five $40 billion banks could be supporting policy conclusions that are exactly wrong. The results more likely are simply showing that larger banks are more resilient than smaller banks and thus tend to fail only when economic conditions are worse — a finding consistent with many other analyses, and a result that favors greater industry consolidation.
[1] To avoid confusion, note that they are reporting a percent change in the growth rate of GDP, not a percent change in GDP or a percentage point change in the growth rate. For example, while they don’t provide enough information to be certain, we think they calculate that quarterly GDP growth falls from 0.8 percent (not at an annual rate) to -0.4 percent (a 150 percent decline) in a quarter in which a $200 billion bank fails.
