On June 25, the Federal Reserve Board released the results of its stress tests for 2020 for both the regular, planned stress test initiated prior to the onset of COVID-19 and a supplementary “sensitivity analysis” in response to the evolving conditions.[1] The results from the regular test show that all large banks remain strongly capitalized, and will be used by the Board to set institution-specific stress capital buffer (SCB) requirements under the new capital rule taking effect in the fourth quarter of this year.
In this note, I review loss rate projections for the major consumer and commercial loan segments from the DFAST 2020 regular test. The analysis compares the projected loss rates from the supervisory models to those from banks’ models and to supervisory projections from prior years (2016 through 2019.)
The analysis demonstrates a systematic tendency for projected loss rates from supervisory models to exceed those from banks’ own internal models. Banks’ models are strictly vetted by examiners from the Federal Reserve and other regulatory agencies and are subject to rigorous validation; they are used to make financial projections for public reporting purposes. Thus, they seem unlikely to include persistent, material errors. A more likely explanation is that the Federal Reserve applies more conservative assumptions or overlays compared to the banks, although what these might be is not clear.
Overall, the discussion highlights a need for greater disclosure and transparency regarding the assumptions of supervisory models. Additional transparency would improve the efficiency of capital allocation across banks’ business lines by reducing uncertainty around capital requirements and lead to lower management capital buffers. Moreover, the public could also benefit from more information on elements of conservatism incorporated into the supervisory models. Presumably, public comment might improve the quality of the supervisory models.
The analysis also characterizes variation in the supervisory projections from year to year. The reasons for the observed patterns also are not clear. Another useful step toward greater transparency would be to provide a decomposition of year-to-year changes in loss rates and loss amounts into (1) changes in banks’ portfolios, (2) changes in the severity of supervisory scenarios, and (3) changes in the supervisory models themselves.
SUPERVISORY VERSUS INDUSTRY MODEL RESULTS
The supervisory models generally yield higher loss rates compared to banks’ internal models, and often these differences are material. A comparison of supervisory versus bank loss rate projections from DFAST 2020, conditional on the Federal Reserve’s severely adverse scenario, is presented in Figure 1. Cumulative credit loss as a percent of average stress period balance is measured along the vertical axis for the supervisory model and along the horizontal axis for banks’ internal models, for institutions subject to both supervisory and company-run stress tests in the 2020 cycle.[2]
Each dot represents an individual institution’s results for one of six loan categories: first-lien residential mortgage; credit card; other consumer; commercial real estate (CRE); commercial and industrial (C&I); and other loans. The diagonal line locates equality between supervisory and bank projections. It is immediately apparent that most of the dots are located substantially above the diagonal line, indicating a systematic tendency for projected loss rates from the supervisory model to exceed those from banks’ internal models.
For the CRE and C&I categories, the supervisory model invariably yields higher loss rates. Within the residential mortgage category, the supervisory loss rate projections from DFAST 2020 materially exceed those generated by internal models for three of the six institutions included in the analysis, while matching the others closely (within a 10 percent range). For credit cards as well for the “other loans” category, the supervisory model also tends to produce higher loss rates compared to banks’ own models.
Figure 1: DFAST 2020 Supervisory versus Bank Projected Loss Rates
The higher loss rates from the supervisory model suggest that the Federal Reserve applies more conservative assumptions in model construction, or more frequently applies conservative adjustments to modeled outcomes, compared to the banks. In some cases, the conservatism might reflect a banks’ use of more granular data compared to the data used for supervisory modeling. For example, if the supervisory data do not include observations associated with certain types of loss mitigation activities, then the supervisory model may not adequately credit a bank whose historical loss performance is persistently better than the industry average because of such activities.
Greater disclosure of such assumptions and overlays would provide policymakers and the public with essential information for understanding the efficacy of the Federal Reserve’s approach. Also, transparency on this dimension is needed so that subject banks can evaluate their own assumptions relative to those incorporated into the supervisory models. Indeed, both the Federal Reserve and the banks would benefit from understanding the differences in their modeling assumptions, potentially enabling both to improve their models.
ANALYSIS OF LOSS RATES UNDER SUPERVISORY MODELS
The remainder of this note examines loss rates projected using the supervisory models over the last five DFAST stress test cycles (2016 through 2020), restricting attention to institutions that participated throughout. For brevity, the analysis focuses on mortgages and consumer loans; similar conclusions apply to the modeled loss rates for commercial real estate and commercial and industrial loans.
The loss rate projections change over time for combinations of reasons, such as changes made to the models along with modifications to the scenarios and shifts in the risk characteristics of the loans. However, the relative importance of these reasons cannot be determined because the modeled outcomes lack transparency. Providing a decomposition of changes in projected loss rates in relation to sources of change would facilitate further research toward a better understanding of the persistent divergence in results.
FIRST-LIEN RESIDENTIAL MORTGAGES
Projected loss rates based on the supervisory (Federal Reserve’s) model for residential mortgages from the 2016 through the 2020 DFAST cycles are shown in Figure 2. Here, attention is restricted to six institutions that currently hold material mortgage portfolios and are active in the mortgage origination market.[3]
These six institutions show a common pattern of declining loss rates based on the supervisory model, with the decline interrupted by the 2018 stress test.[4] Also, over time the differences across institutions narrow, and by 2020 there is less than 1 percentage point differential between the institutions with the lowest and highest projected loss rates.
The moderating projected loss rates are consistent with empirical indicators of mortgage credit risk, including with observed “roll” or transition rates of mortgage balances from current to seriously delinquent, which have declined continually since 2015.[5] They can be explained by appreciating home values during this period, as reflected in the Federal Housing Finance Agency (FHFA) house price index, represented by the dotted line superimposed on Figure 2. Rising home values translate into lower loan-to-value ratios on existing mortgage loans, which in turn reduce the risk of default of these mortgages. The relative convergence of loss rates by 2020 also may reflect appreciating home values, which would tend to mitigate the effects of the risk factors contributing to differences across institutions in loss exposure.
Figure 2: Projected Loss Rates for First-Lien Mortgages (% of average balances)
The uptick in projected stress-period loss rates in the 2018 DFAST can be attributed to the relatively severe scenario for residential real estate values specified in the supervisory stress scenario that year. Stressed loss rates did not continue to decline between 2019 and 2020 despite the continued appreciation in home values, which may reflect a marginal increase in the severity of the economic scenarios.[6]
Despite the consistency of this series of projected loss rates with observed risk trends, transparency regarding the results is insufficient. One key issue that bears repeating is that the supervisory loss rate projections from DFAST 2020 materially exceed those generated by bank internal models for three of the six institutions represented in Figure 2 while matching closely for the others. A related observation is that the Federal Reserve’s stressed loss rate projections for banking institutions’ mortgage portfolios are also relatively high when compared to loss rate projections from stress tests of the government-sponsored institutions (GSEs) in the secondary market mortgage, Fannie Mae and Freddie Mac.
The Federal Housing Finance Agency, which supervises the GSEs, since 2014 has required them to conduct annual stress tests pursuant to the Dodd-Frank Act.[7] Whereas projected, stress period loss rates for 2019 and 2020 from Figure 2 range from 1.2 to 1.6 percent, the projected credit loss as a percent of the average balance for the combined GSEs from their 2019 stress tests is about 0.2 percent (2020 results are not yet published.)[8] There are no differences between the economic scenarios employed by the Federal Reserve Board and the GSEs that might explain these differing outcomes.
These differences may reflect different assumptions applied in modeling mortgage borrower performance or differences in the data used for modeling (the extent to which the data used to develop the Federal Reserve’s model differs from GSE data.) More transparency from the Fed about its models would facilitate use of the GSE results as a benchmark for the Federal Reserve’s model, by enabling a more informed comparison.
It would also be helpful to know how dependent the Fed’s modeling is on data from the mortgage crisis period of 2008 through 2011 because many of the conditions affecting mortgage losses during that historical period no longer apply. For example, mortgage lenders including banks are far more cautious in extending credit than they had been during the run-up to the mortgage crisis, in part because of new consumer protection regulations such as the Consumer Financial Protection Bureau’s Ability-to-Repay (ATR) and Qualified Mortgage (QM) rule mandated by the Dodd-Frank Act.
The ATR and QM rule require that lenders make a reasonable and good faith determination, based on verified and documented information, that the consumer has a reasonable ability to repay the mortgage loan. The rule’s ability-to-repay standards apply primarily to loans that are not government-insured or intended for sale to Fannie Mae or Freddie Mac and for which the borrower’s monthly debt payments would exceed 43 percent of monthly income (non-QM loans.) The Bureau’s 2019 assessment report on the ATR and QM rule finds evidence of improved repayment performance of non-QM loans tied to these standards.[9]
Another factor potentially mitigating mortgage credit risk relative to the historical period is the stronger financial condition of households as of the beginning of 2020. In particular, the financial obligation ratio calculated by the Federal Reserve, which measures household financial obligations including housing (mortgage or rent) and consumer debt service payments as a percentage of disposable personal income, has declined from 18.0 as of the third quarter of 2007 to 15.0 as of the first quarter of 2020.[10] Also, according to the most recent (2016) Survey of Consumer Finances, the inflation-adjusted, median value of financial assets of homeowners increased from 63.8 to 69.2 thousand dollars over this period.[11]
In the stress testing context, it is important not only for the Federal Reserve to understand the limitations of its supervisory models, but also to share that understanding with subject banks, policymakers, and the public so that they can make the best use of the results. In particular, policymakers and the public need to understand the ways in which the loss rate projections are not forecasts conditional on scenarios. As explained in the regulatory agencies model risk management guidance (SR 11-7):
Decision makers need to understand the limitations of a model to avoid using it in ways that are not consistent with the original intent. Limitations come in part from weaknesses in the model due to its various shortcomings, approximations, and uncertainties. Limitations are also a consequence of assumptions underlying a model that may restrict the scope to a limited set of specific circumstances and situations.[12]
CREDIT CARDS
Projected, stress period cumulative loss rates for bank credit card portfolios from the 2016 through 2020 DFAST cycles are shown in Figure 3.[13] All eight institutions exhibit an accelerated rise in projected loss rates after 2018, and most exhibit rising loss rates through the entire series of stress tests. The extent of heterogeneity across institutions has remained fairly consistent throughout the series of stress tests. The upward trend in projected, stress period loss rates is consistent with observed “roll” or transition rates of card balances from current to 90 days or more past due, which were increasing during most of this period.
Figure 3: Projected Loss Rates for Card Exposures (% of average balances)
However, the Federal Reserve Board made changes to the credit card stress test model prior to the 2018 DFAST cycle, which reportedly contributed to the increases in projected losses between 2017 and 2019.[14] An additional consideration is that the unemployment scenarios for the stress tests became progressively more severe in that they incorporated the larger increases required to maintained peak, stress period unemployment rates at 10 percent.[15]
Thus, greater transparency is needed around the relative contribution of model changes and scenario effects versus ex-ante risk composition of the borrower population to loss rate projections. Providing transparency along this dimension would enhance the value of the stress tests by informing subject banks and the public on market risk trends. Moreover, absent such transparency, it is difficult for policymakers and the public to assess the ability of the supervisory models to monitor risk and to thereby sustain confidence in the models.
As with the first-lien mortgage model, the public would benefit from the Federal Reserve providing analysis around potential limitations of the model, such as the degree to which the model captures evolving conditions in the credit card market. For instance, the Credit Card Act of 2009 (the Card Act), which was implemented in early 2010, addressed practices in setting credit card rates and terms that were non-transparent and conducive to market inefficiencies. Changes in issuer practices mandated by the Card Act have helped to make card pricing more transparent to consumers, potentially ameliorating credit risk tied to consumer miscalculations in borrowing decisions.[16]
Some researchers have argued that many consumers are subject to behavioral biases that may lead them to accumulate too much debt, and that lenders in the past may have found it profitable to indulge this tendency (via lax credit standards for issuing new cards or providing line increases.)[17] The Card Act’s limitations on over-limit fees, late payment fees, and interest rate increases likely made any such indulgence less profitable, thus likely helping to mitigate potential impacts of consumers’ behavioral biases on their risk of default.
Alongside changes in industry practices resulting from the Card Act, banks’ risk management approaches have likely evolved as well. Bank decisions on originating new accounts or granting line increases are based on revised models relying on new data, which are likely to reduce delinquencies and defaults in a future downturn.
Thus, in the card context as in the mortgage context, statistical modeling of credit loss is unlikely to fully account for the changing circumstances. Assessing the uncertainty around the loss projections for credit cards requires an understanding of such limitations.
OTHER CONSUMER CREDIT
In the interest of brevity, I will simply note that projected loss rates for non-mortgage, non-card consumer loans exhibit modest variation over time. They range widely across institutions, with the rank ordering across institutions generally stable.
This loan category also is subject to significant transparency concerns. The heterogeneity of this segment, which incorporates private student loans, auto loans, generic consumer installments loans, and small business loans with less than 1 million dollars in committed balances, imparts a significant degree of opaqueness to the modeled outcomes. The lack of transparency is compounded by changes made to the auto loss model prior to the 2019 DFAST cycle, which reportedly had a material effect on the 2019 and 2020 loss projections.
Thus, more transparency is needed around the relative contribution to of loan mix, model changes and scenario effects versus ex-ante risk composition of the borrower population to loss rate projections. In addition, as with other categories, some discussion of model limitations would be appropriate.
CONCLUSION
This post has examined loss rate projections for the first-lien mortgage and consumer loan segments from DFAST 2020. Results show that projected loss rates from the supervisory models consistently exceed those obtained using banks’ own models.
Since banks’ models are strictly vetted by examiners from the Federal Reserve and other regulatory agencies on the basis of standards established by the agencies, it is unlikely that banks’ models inadequately capture risk. Rather, the higher loss rates from the supervisory model suggest that the Federal Reserve applies more conservative assumptions or overlays compared to the banks, although what these might be are neither clear nor transparent.
In addition, the note compares the evolution of loss rates reported in the supervisory stress test between 2016 through 2020. Those results show that banks and the public would benefit from increased transparency regarding the relative importance of portfolio changes, changes in the severity of supervisory scenarios, and changes in supervisory models as sources of the year-over-year change in projected loss rates. Greater transparency along this dimension would enhance the value of the stress tests to subject banks and the public by providing information about market risk trends, and it would help maintain the public’s confidence that the stress tests reliably monitor risk.
With respect to both the nature of conservative adjustments and the changes in loss rates, banks need transparency in order to make effective capital budgeting and strategic decisions. Uncertainty about the capital requirement, including a lack of information regarding the source of conservative adjustments to loss models, leads banks to maintain larger management buffers above and beyond regulatory buffer requirements. While this arguably makes banks safer, it reduces their capacity to extend credit, especially the types of credit for which there is the greatest uncertainty regarding the supervisory models.
Likewise, by having more transparency around how macroeconomic scenarios map to losses and ultimately to capital requirements, banks will be able to make better decisions on how to allocate capital across the different business lines in the medium- and long-term. Moving into a new loan segment is not a decision a bank makes lightly (nor, conversely, is eliminating lending to a certain loan segment) and there are sunk costs associated with these decisions.
The analysis also highlights how the public could benefit from more discussion of potential limitations of those models and the associated uncertainty around model projections. As the regulatory agencies have emphasized in their guidance on model risk management: “Models are never perfect… In all situations, it is important to understand a model’s capabilities and limitations given its simplifications and assumptions.” In the stress testing context, it is important not only for the Federal Reserve to have this understanding in regard to the supervisory models, but also to share as much as possible of that understanding with subject banks, policymakers and the public in order that they make the best use of the results.
[1] The original scenarios for the 2020 stress testing cycle were released by the Federal Reserve Board on February 6, 2020, as described here: https://www.federalreserve.gov/newsevents/pressreleases/bcreg20200206a.htm. The published report summarizing the DFAST 2020 supervisory stress test results may be viewed at: https://www.federalreserve.gov/publications/files/2020-dfast-results-20200625.pdf
[2] Institutions with relatively small total exposure amount or that have runoff portfolios (due to a strategic exit decision) in a given loan category are excluded.
[3] Primarily non-retail institutions (GS, MS, NT, and BNY) are excluded. In addition, I choose to exclude Citigroup, COF, and HSBC because of their strategic decisions to exit the mortgage business and sell-off portions of their portfolios during this period.
[4] Even steeper declines in modeled loss rates occurred through the initial set of stress tests between 2012 and 2016, due to improving economic conditions and declining inventory of the higher-risk mortgages that were originated prior to 2010.
[5] This roll rate is the percentage of outstanding balances that were non-delinquent in the previous quarter and are 90 days or more past due in the current quarter. For the observed, quarterly time series of these transition rates, see https://www.newyorkfed.org/medialibrary/interactives/householdcredit/data/pdf/hhdc_2019q4.pdf
[6] The peak-to-trough decline in home prices was a bit more negative in DFAST 2020 compared to 2019. Also, the market interest rate decline in the severely adverse scenario is smaller for 2020 compared to 2019 (due to very low-interest rates at the start of 2020.) Lack of a material interest rate decline limits the ability of households to benefit from reduced mortgage payments through refinancing.
[7] As amended by Section 401 of the Economic Growth, Regulatory Relief, and Consumer Protection Act, the Dodd-Frank Act requires certain financial companies with total consolidated assets of more than $250 billion, and which are regulated by a primary federal financial regulatory agency, to conduct periodic stress tests to determine whether they have sufficient capital to absorb losses and support operations during adverse economic conditions.
[8] See https://www.fhfa.gov/AboutUs/Reports/ReportDocuments/2019_DFAST_Severely-Adverse-Scenario.pdf
[9] The report finds that the performance of non-QM loans improved relative to the performance of comparable QM loans. See https://files.consumerfinance.gov/f/documents/cfpb_ability-to-repay-qualified-mortgage_assessment-report.pdf.
[10] These figures are seasonally adjusted. See https://www.federalreserve.gov/releases/housedebt/default.htm.
[11] Assets are measured in 2016 dollars. In addition, the percent of homeowners with a home equity line of credit declined from 12.4 to 6.9 between 2007 and 2016, implying that mortgage borrowers have larger equity stakes in their homes, mitigating default risk. See https://www.federalreserve.gov/econres/files/BulletinCharts.pdf.
[12] See https://www.federalreserve.gov/supervisionreg/srletters/sr1107a1.pdf
[13] I choose to exclude HSBC since it had a runoff portfolio until 2018, when it made a strategic decision to re-enter the credit card market in the U.S.
[14] These changes included refinements to the model along with re-estimation of model parameters using additional data. An additional refinement and re-estimation was implemented prior to 2020, but the affect on modeled outcomes reportedly was more limited. The changes are discussed in the Board’s 2018 model change disclosure letter (https://www.federalreserve.gov/supervisionreg/files/model-change-letter-20180302.pdf) and in the Board’s 2019 and 2020 DFAST summary reports: https://www.federalreserve.gov/publications/files/2019-dfast-results-20190621.pdf and https://www.federalreserve.gov/publications/files/2020-dfast-results-20200625.pdf.
[15] The U.S. unemployment rate declined from about 5 to 4 percent between January 2016 and January 2018, and it fell by an additional half percentage point between January 2019 and January 2020.
[16] Importantly, in a 2013 study the CFPB determined that the Card Act’s “limitations on ‘back-end’ fees, along with restrictions on an issuer’s ability to raise interest rates, have simplified a consumer’s cost calculations. Credit card costs are now more closely related to the clearly disclosed annual fees and interest rates. This greater transparency means a consumer deciding whether to charge a purchase can now make that decision with far more confidence that costs will be a function of the current interest rate rather than some yet-to-be-determined interest rate that could be reassessed at any time and for any reason by the issuer.” See https://files.consumerfinance.gov/f/201309_cfpb_card-act-report.pdf
[17] See Sumit Agarwal and Jian Zhang (2015), “A Review of Credit Card Literature: Perspectives of Consumers,” for a review the scholarly literature on the role of consumer behavioral biases in consumer credit markets, focusing on three biases that have received the most attention in this literature. These biases imply that consumers may make borrowing decisions that are not in their overall or long-term best interest. The study is available at https://www.fca.org.uk/publication/market-studies/review-credit-card-literature.pdf
