The use of machine learning (ML) models, especially when applied in combination with alternative credit data, can improve credit decisions and facilitate broader access to credit. ML combined with creative use of alternative data sources have helped propel the expanding market share of fintech companies in consumer and small business lending.
Not only fintechs, but also many banks have been exploring the use of ML and alternative data for consumer and small business credit decisions. In doing so, banks consider not just the possible benefits but also the potential risks and costs associated with the new approaches, alongside compliance with regulatory and supervisory standards.
Because fintech companies are subject to much less regulation and supervision than banks, they face fewer limitations on their ability to implement new credit models and decision processes. This disparity puts fintechs and banks on a different regulatory playing field that creates a competitive advantage for fintechs around operating and compliance costs. It also leaves the customers of fintech companies relatively unprotected from potentially harmful or discriminatory behavior.
This post briefly reviews how ML models and alternative data can play a role in broadening household and small business access to credit and describes the cost and risk tradeoffs associated with their use. The discussion highlights the need for regulatory policies and practices to keep pace with the developments in credit modeling and to apply a consistent approach for banks and fintechs, to ensure equitable treatment of all borrowers under the consumer protection laws.
Scoping the Role of ML
Machine learning methods of data analysis are characterized by powerful capability to automate selection of explanatory variables and evaluate complex interactions among the variables. In modeling borrower and loan outcomes, such as likelihood of delinquency, ML approaches are most useful when there are many combinations of factors potentially predictive of borrower repayment performance.[1]
Modeling repayment performance of consumer loans is distinctly challenging because available data typically do not include direct measures of household cash flow and liquidity. Household income and liquid assets are self-reported and unverified, if observed at all; many ongoing expenses are not observed; spending and savings habits and employment stability are not directly observed.
Traditional, generic consumer credit scoring models rely on credit history variables as indirect indicators for quantifying consumers’ risk of cash flow shortfalls. These models have had a long history of development and have been validated in numerous lending contexts and under varying macroeconomic environments. Given the proven robustness of the traditional credit scoring approaches, it is not a given that ML will yield material, predictive improvements in all situations. However, there may be important applications in which ML based models provide significant lift over traditional scoring models, by better capturing the underlying cash flow fundamentals or consumer behavioral factors that affect repayment performance.[2]
A major impediment to making credit more accessible and affordable to lower income individuals and households is the comparatively high frequency of flawed or limited credit histories or below prime credit scores among this population. Another common impediment is adverse selection: the phenomenon whereby within an observably homogeneous population, consumers that accept an offer of credit often may be riskier than those that turn down the offer.[3] In general, lenders recognize the potential for adverse selection ex-ante and to the extent that it is present may be less willing to offer credit (for instance, may reduce the offered credit limits or apply stricter qualification criteria) or may compensate for the risk by requiring a higher interest rate.
ML may be particularly useful in such contexts. ML modeling may support marketing strategies that reach a broader population, mitigating adverse selection through improved identification of delinquency risk.[4] For example, enhanced risk differentiation may facilitate offering lower APR card products or more generous balance transfer terms to low- or moderate-income households that present lower risk of delinquency. Similarly, ML based modeling may enable expanded small business credit offerings.
Promising Uses of Alternative Data
Ultimately, the effectiveness of a model for predicting delinquency or other aspects of loan profitability is determined by the data on which it is based. This simple fact has two important implications for ML modeling.
First, the model risks and compliance costs associated with developing next generation ML models may not be worth bearing if not offset by additional benefits from introducing new data. Simply applying a more flexible or powerful data analysis tool to the same set of data does not necessarily provide materially improved prediction of borrower or loan outcomes. The gains achievable from applying ML tend to be more substantial when building a next generation model based on an expanded set of variables to consider.
New data, of course, may encompass either databases not previously explored, or new and creative uses of existing data sources, or both. Improvement may be tied to the new data better reflecting underlying cash flow fundamentals or from more fully capturing situational or consumer behavioral factors influencing debt repayment.
Second, the introduction of new data requires careful consideration of data quality, consistency, and suitability. ML models estimated on inaccurate data, or on data that are not representative of the population to which the model is to be applied, can yield unreliable predictions, just as with traditional models.
There is an emerging consensus that the use of alternative payment history data, such as rent and utility payment histories, can enhance risk differentiation, especially for population groups with thin credit files or below prime credit scores.[5] For example, Di Maggio, Ratnadiwakara, and Carmichael (2022) apply ML modeling to anonymized administrative data provided by a major fintech platform to investigate the lift provided by alternative data, including such as education and employment history, to assess borrowers’ creditworthiness. They find that compared to a traditional scoring model, use of the ML based model with alternative data results in significantly reduced probability of being rejected and lower interest rates for those approved.[6]
Furthermore, a growing, behavioral finance literature points to the importance of behavioral factors in determining consumer borrowing and repayment performance. Potentially relevant categories of behavioral variables include debt payment prioritization patterns; purchasing and payment habits; financial literacy factors; influence of behavioral biases such as present bias; and attentiveness (including responsiveness to “nudges”). Expanding the set of explanatory variables to incorporate such factors may permit models that provide a more accurate picture of borrowers’ repayment performance.[7]
Risk and Cost Tradeoffs
The potential benefits of ML applications and alternative data must be balanced against costs related to model risk management, model validation, and regulatory compliance. All model development activity entails such costs, but the relative complexity and non-transparency of ML models and more limited experience with using nontraditional data exacerbates these costs.
When an established model with demonstrated reliability (conditional on periodic, minor updating) through varying economic and market environments is replaced, stability concerns naturally arise, given that experience with the new model is limited. Of course, stability concerns necessitate greater attention to model risk management and model validation.
If the replacement model is ML based and as such is complex and relatively opaque, stability concerns are heightened. To the extent that the fitted relationships are difficult to evaluate intuitively and judgmentally, it becomes more difficult to assess conceptually whether the model has been overfitted to the development data, in which case it is apt to be unstable.[8]
Likewise, replacement of an established model that has been previously vetted for regulatory compliance purposes entails compliance risks, encompassing the areas of fair lending, adverse action explainability, and model risk management. Again, ML based models and reliance on alternative data sources can raise heightened concerns, because of the opacity or potentially vague interpretability of the models or data. For instance, it may be more difficult to demonstrate absence of disparate impact discrimination when opaque interactions exhibit correlation with protected characteristics.
As emphasized in Bank Policy Institute (2020), the federal banking agencies should consider ways to mitigate the litigation and enforcement risk that banks face in adopting ML based models or using nontraditional data. For example, appropriate consideration should be given to the use of modern techniques for detection and mitigation of algorithmic bias, and to evolving approaches to developing “explainable” ML models and to validating ML based models.[9] Moreover, as expressed in Bank Policy Institute (2021), regulators should “avoid creating or applying new regulatory expectations that may hinder progress in using this evolving technology” and should allow banks flexibility in managing the model risks attendant to use of ML approaches.
The Non-Level Regulatory Playing Field and its Shortcomings
Although fair lending and consumer financial protection laws apply to both bank and nonbank lenders, they have been differentially enforced. Whereas “most non-bank lenders are not regularly examined by any federal (or state) agency, banks are examined on a regular basis, in many cases by multiple agencies, and larger banks have on-site examination teams providing constant supervision.”[10]
The CFPB has the authority to bring enforcement actions against banks and non-banks and examination authority over both large banks and certain types of nonbank lenders (“larger participants”).[11] However, in practice, most nonbank lenders tend to face limited fair lending examination and enforcement from the CFPB, and state-level fair lending oversight and enforcement varies widely.[12] Thus, “even non-bank lenders operating on a national scale have been subject to limited and uneven scrutiny of their fair lending practices as compared to banks.”
Model risk management is another important area where banks and nonbanks receive different treatment from regulators. Federally regulated banks and other depository institutions are subject to the interagency Model Risk Management Guidance, while non-bank lenders face no comparable oversight of or limitations on model development and use.
Banks have a long history of compliance with the Model Risk Management Guidance and are regularly supervised by prudential regulators. In contrast, nonbanks engaged in financial services are not subject to model risk management or regulatory compliance standards and are not regularly or consistently supervised by prudential regulators.
This non-level regulatory playing field puts banks at a competitive disadvantage. It reduces the cost of consumer and small business lending for fintechs relative to banks and eases the way for fintechs to introduce new products and target new markets.
Ultimately, to the extent that lending activity shifts from regulated to unregulated institutions, the efficacy of regulation is undermined and market efficiency (which presumably depends on effective regulation) is hampered. As fintechs gain market share, consumers are less assured of fair lending and other financial protections. Moreover, if fintechs are insufficiently attentive to model risk, excessive credit may be extended to financially at-risk households, harming those households in the long run when they face difficulties repaying the debt.
Furthermore, fintech lending may increase household debt burdens, potentially compromising consumers’ financial health while increasing banks’ risk exposure indirectly. Recall that both competitive pressures on banks and indirect effects on their risk exposure were evident during the nonbank, subprime mortgage lending boom that precipitated the mortgage crisis in 2007.
Consumers should be equally protected regardless of what kind of entity they engage with for financial services. As enunciated in Bank Policy Institute (2020), the regulatory framework around use of ML and alternative data in credit underwriting must apply equally to banks and non-banks. Consumers stand to benefit when the regulatory environment grants banks and nonbanks the same opportunities to innovate, while simultaneously ensuring borrowers enjoy a consistent level of consumer protections whether they borrow from a bank or a nonbank fintech.
[1] Nonlinearity is present when a relationship varies in direction or magnitude across the range of relevant values of the explanatory variable.
[2] Khandani, Kim, and Lo (2010), for instance, find that ML approaches significantly improve the classification rates of credit-card-holder delinquencies using combined customer transactions and credit bureau data from January 2005 to April 2009 for a sample of a major commercial bank’s customers. Butaru et. al (2016) find that application of ML techniques improves model performance for risk management of credit card accounts, including for decisions around credit line increases or decreases.
[3] For example, they may be facing an unexpected rise in expenses or reductions in income, not yet reflected in their credit record. Some may proceed to become over-extended, such as by adding rather than transferring balances in the context of a balance transfer offer.
[4] Conversely, ML is likely to be less useful when the goal is to develop broad classifications.
[5] See, for instance, Urban Institute (2021).
[6] The borrowers most positively affected are the those with low credit scores and short credit histories but also a low propensity to default.
[7] For instance, Donnelly et al. find, based on a large field experiment, that the repayment behavior of credit card borrowers may depend on the payment options offered in monthly billing communications. Most notably, borrowers who were given the opportunity to allocate their payment toward specific purchase categories paid down their debt balance more quickly than a control group. Lee, Yang, and Anderson (2021) show that accounting for card holders’ grocery shopping habits can enhance the predictive performance of credit card payment models. The study identifies five broad habits that are correlated with credit card payment behaviors: (1) shopping the same day of week, (2) spending similar amounts on each trip, (3) consistently buying the same brands and categories, (4) taking advantage of deals and promotions, and (5) buying healthier products.
[8] Wang and Perkins (2021) provide a case study of the stability of ML based models, using a sample of personal loans from a large fintech company. The study finds that that use of ML “improves default predictions much more for in-sample estimates than out-of-sample estimates” and that deteriorating relative performance of the ML based models eventually leads them to underperform a traditional, logistic model. The study concludes that “caution is needed in applying ML methods, especially over a business cycle.”
[9] For recent, in-depth discussion of these new and evolving approaches, see Fu, Huang, and Singh (2020), Bakkar et. al (2021), and Krivorotov and Richey (2022).
[10] See Bank Policy Institute (2020).
[11] Id.
[12] Id.
References
Bank Policy Institute, “Artificial Intelligence: Recommendations for Principled Modernization of the Regulatory Framework.” (October 2020). Artificial-Intelligence-Recommendations-for-Principled-Modernization.pdf (bpi.com)
Bank Policy Institute, Response to Interagency Request for Information and Comment on Financial Institutions’ Use of Artificial Intelligence, including Machine Learning (Docket No. OCC-2020-0049; OP-1743; RIN 3064- ZA24; CFPB 2021-0004; NCUA 2021-0023). (June 25, 2021). BPI-Comment-Letter_Interagency-RFI-on-AI_Final-06.25.2021.pdf
Bakkar, Imane, Chiranjit Chakraborty, Carsten Jung, Marta Kwiatkowska and Carl Taylor, “Software Validation and Artificial Intelligence in Finance – a Primer.” Bank of England, Staff Working Paper No. 947 (October 2021). Bank of England Staff Working Paper No. 947
Butaru, Florentin, Qingqing Chen, Brian Clark, Sanmay Das, Andrew W. Lo, and Akhtar R. Siddique, “Risk and Risk Management in the Credit Card Industry.” Journal of Banking and Finance 72, 218-239 (August 2016). Risk and risk management in the credit card industry (uchicago.edu)
Di Maggio, Marco and Dimuthu Ratnadiwakara, and Don Carmichael, “Invisible Primes: Fintech Lending with Alternative Data.” (May 28, 2022). Available at SSRN:
https://ssrn.com/abstract=3937438
Donnelly, Grant E., Cait Lamberton, Stephen Bush, Zoe Chance, and Michael I. Norton, “Repayment-by-Purchase Helps Consumers to Reduce Credit Card Debt.” Harvard Business School Marketing Unit Working Paper No. 21-060 (November 6, 2020). Available at SSRN: https://ssrn.com/abstract=3728254
Fu, Runshan, Yan Huang, Param Vir Singh, “AI and Algorithmic Bias: Source, Detection, Mitigation and Implications.” (July 26, 2020). Available at SSRN: https://ssrn.com/abstract=3681517
Khandani, Amir E., Adlar J. Kim, and Andrew W. Lo, “Consumer Credit Risk Models via Machine-Learning Algorithms.” Journal of Banking and Finance 34(11), 2767-2787 (November 2010). Consumer credit-risk models via machine-learning algorithms – ScienceDirect
Krivorotov, George and Jeremiah Richey, “Explaining Denials: Adverse Action Codes and Machine Learning in Credit Decisioning.” (June 10, 2022). Available at SSRN: https://ssrn.com/abstract=4133915
Lee, Jung Youn, Joonhyuk Yang, and Eric Anderson, “Buying and Payment Habits: Using Grocery Data to Predict Credit Card Payments.” (June 16, 2021). Available at SSRN: https://ssrn.com/abstract=3868547
Urban Institute, “Adopting Alternative Data in Credit Scoring Would Allow Millions of Consumers to Access Credit.” Urban Wire (March 15, 2021). Adopting Alternative Data in Credit Scoring Would Allow Millions of Consumers to Access Credit | Urban Institute Wang, J. Christina and Charles B. Perkins, “How Magic a Bullet Is Machine Learning for Credit Analysis? An Exploration with FinTech Lending Data.” (October 21, 2019). Available at SSRN: https://ssrn.com/abstract=3928076
