A recent Federal Reserve staff paper by Lorenc and Zhang (2018) (henceforth LZ) looks at the relationship between bank failures on real economic activity and finds that the impact of bank failures on economic activity increases sharply with bank size. In particular, they find that the economic impact of the failure of a single large bank is three times greater than the simultaneous failure of five banks that are each one fifth the size of the large bank. Their extraordinary result suggests that regulation and supervision should become exponentially more stringent as bank size increases.
However, we find that their result is weak and does not hold up to standard robustness checks.
It is well accepted that the failure of a systemically important bank has a disproportionate impact on economic activity relative to a non-systemic institution. But the use of size to determine the systemic importance of a bank is not proven by this paper.
In our critical assessment of their paper, we find that the impact of financial stress of large banks on real outcomes is statistically weak even under their baseline specification. In addition, the statistical significance of their findings vanishes when we conduct robustness exercises akin to those performed in Bernanke (1983), which the LZ paper claims to build upon. Specifically, this post assesses the robustness of their findings by conducting the following analysis:
1. Following Bernanke (1983), we use the change in deposits of failed banks instead of the natural logarithm as the transformation to the key explanatory variables. Using the Bernanke specification, we find the impact of large bank failures on real GDP growth is almost never statistically different from zero at conventional levels.
2. We use alternative proxies to control for the decline in economic activity during recessions and assessed the robustness of the statistical importance of deposits of failed large banks. In particular,
a. We add dummy variables for each quarter during the 2008 financial crisis to control for other factors that might have explained the sizable decline in real GDP growth during the past financial crisis.
b. We include other macroeconomic variables that constitute an alternative proxy for the 2008 financial crisis, such as the growth rate in the house price index and the change in corporate bond spreads.
In each case, the impact of large bank failures on real GDP growth is never statistically different from zero at conventional levels.
3. We show that the regressions in the paper fail to correct for heteroskedasticity and autocorrelation in the residuals. As a result, the estimated standard errors are incorrect and lead to an overstatement of the statistical significance of the estimated coefficients.
To arrive at their results, LZ use linear regressions to estimate the impact of bank failures on two measures of economic activity: growth of real GDP and change in the unemployment rate. To show that the failure of a large bank has a stronger negative impact on economic activity relative to the failure of smaller banks, the paper assesses the economic and statistical significance of the coefficients associated with deposits of failed banks for different percentiles of the distribution of bank assets. In our analysis as in the LZ paper, bank size is represented by the top percentiles of the bank size distribution each quarter. For brevity, we only report the results for real GDP growth.
First, the figure above shows the estimated impact of an increase in deposits of failed banks (in $ trillions) on real GDP growth. The only difference relative to LZ is that it uses the dollar amount of of failed banks instead of the natural logarithm of deposits of failed banks. The specification that uses the levels of deposits instead of the natural logarithm is the specification that is the closest to Bernanke (1983). The solid dots denote coefficients that are statistically different from zero at the 5 percent level while the hollow dots represent estimated coefficients that are not statistically different from zero. As shown in the chart, only the coefficient associated with the 0.5 top percentile on bank size is statistically different from zero at the 5 percent level. Also, the sensitivity of real GDP growth to bank failures no longer depends on bank size. The line that connects the estimated coefficients is about flat around -0.5 except for the 0.1 and 0.5 top percentiles.
 |
 |
Next, the two charts above depict the estimated coefficients by including alternative proxies for the financial crisis in each regression. As shown by the hollow dots in the two charts, the impact of deposits of failed large banks on real GDP growth is no longer statistically different from zero at the 5 percent level. The specification depicted on the left panel, includes as additional explanatory variables dummy variables for each quarter from 2008:Q1 and 2009:Q2. The motivation for these alternative specifications is that there are other factors that were important in explaining the decline in real GDP growth during the Great Recession besides bank failures. Bernanke (1983) conducted a similar experiment and found that the magnitude and statistical significance of the coefficient on the dollar amount of failed banks during the Great Depression increased after the inclusion of the dummy variables. In contrast, we find the opposite using the specification chosen by LZ. Note that we used the specification in natural logs as in LZ, since, as we showed above, the relationship between large bank failures and real GDP growth is weaker in levels. The panel to the right, replaces the dummy variables with the quarterly growth of the house price index (lagged one quarter) and the change in the corporate BBB spread (also lagged one quarter). The inclusion of other macroeconomic variables is another way to proxy for the direct causes of the past financial crisis. As is the case for the inclusion of dummy variables, the relationship between large bank failures and real GDP growth is no longer statistically different from zero as shown by the hollow dots in the chart.
 |
 |
Regarding our third point, the charts above show the estimated impact of a 1 percent increase in deposits of the largest failed banks on real GDP growth under the baseline assumptions of the LZ paper. The estimated coefficients – represented by the solid and hollow dots – are very close to the ones reported in Table 1 of the LZ paper (p. 13). The main difference between our results and those reported in the LZ paper concerns the statistical significance of the coefficients. As shown by the blue dots on the left chart above, LZ find that the estimated coefficients are highly significant (at the 1 percent level) for bank size percentiles above 1 percent. However, as shown in the chart to the right, we find that none of the coefficients are statistically different from zero at the 1 percent level, and just the 0.2, 0.3, 0.4 and 0.7 percentiles are statistically different from zero at the 5 percent level (denoted by the solid red dots). To generate the confidence intervals on the right panel, we used Newey-West standard errors, which account for autocorrelation and heteroskedasticity of the residuals. 1 The paper does not specify which type of standard errors were calculated to generate the results in Table 1 of LZ but it appears likely that they did not correct for the presence of heteroskedasticity and autocorrelation in the residuals of the model, leading to an overstatement of the statistical significance of the coefficients.
In conclusion, and as noted earlier, these results should not be interpreted as suggesting that the failure of a systemic bank causes the same impact on economic growth as the failure of a smaller non-systemically important bank. Instead, these results indicate that the results of the LZ paper should not be used to inform supervisory policy or regulatory design and that bank size is not a strong determinant of the systemic risk posed by the failure of a large bank.
1 The models include the lagged dependent variable, thus the problem is mostly due to the presence of heteroskedasticity in the residuals.
