As the incoming Administration looks for ways to rationalize regulation and promote economic growth, one obscure but important candidate emerges: the federal banking agencies’ Model Risk Management Guidance. It is a set of check-the-box instructions on how banks should validate and document the models they use across all aspects of their operations. It has disrupted bank operations; created a massive compliance bureaucracy; hampered banks’ efforts to make effective use of AI; and impeded banks from innovating in everything from lending to anti-money laundering monitoring to cybersecurity. Its application has also varied greatly from examiner to examiner. Moreover, that manual is now 14 years old, and has never been updated to reflect the revolution in computer analysis that has occurred since its adoption.
Also, it is being enforced illegally. Rescission therefore appears to be required under Executive Order 14219.[1]
The Model Guidance should be withdrawn and re-proposed for public comment. There is a legitimate role for bank examiners to play in assessing how banks manage some of their models, and clear guidance for them could be useful in ensuring consistency in their work. But the scope of that guidance should be limited to models that are producing outcomes that are material to the bank’s condition or financial reporting. Its interaction with AI and Gen AI needs to be considered. And it needs to allow for banks to pilot and test new models without fear of sanction.
Of course, even for systems not covered by any new guidance, banks would of course continue to backtest and validate the models they use. The question then becomes: in what areas is it necessary for the government to oversee the modeling done by a bank in a way that it does not oversee the modeling of any other company, financial or non-financial?
Illegal Application
In 2011, the OCC and the Federal Reserve Board jointly issued Supervisory Guidance on Model Risk Management. The FDIC adopted the guidance in 2017.
In 2019, the Government Accountability Office ruled that, while termed “guidance,” the 2011 issuance was a rule for purposes of the Congressional Review Act. Because it was never submitted to Congress for its approval, it was therefore invalid and non-binding. The agencies have never submitted the guidance since the 2019 ruling.
Nonetheless, the banking agencies have continued to enforce the guidance as a binding rule. Deviation from the Model Risk guidance is grounds for a Matter Requiring Attention — an examiner mandate that operates as an informal enforcement action — and ultimately a ratings downgrade. Any Matter Requiring Attention requires senior management and board consideration. Thus, compliance and audit staff at banks treat the guidance as a binding rule.
The “Guidance”
The guidance is a 21-page instruction manual from the government on how banks must manage models in their operations. (Readers are strongly encouraged to use the link above and read a random page or two for context.) In addition, the OCC has issued a 109-page Comptroller’s Handbook that provides further details on what is required by the interagency guidance; OCC examiners require adherence to the handbook to the same extent as the guidance. Every bank has been forced to develop procedures to implement all applicable guidance or handbooks; then to establish both compliance and audit functions to confirm that those procedures are followed for each model at the time it is developed and periodically thereafter; and then in many cases to consult examiners on whether they are comfortable with the model or a change to it. There are thousands, perhaps tens of thousands, of people in the banking industry whose job it is to confirm compliance with the Model Risk guidance.
The Model Risk guidance was originally targeted at models that generated significant outputs to measure the bank’s financial condition. The original examples given for its application were “underwriting credits; valuing exposures, instruments, and positions; measuring risk; management and safeguarding client assets; and determining capital and reserve adequacy.” These are areas where models engage in complex calculations to produce material quantitative outputs. Over time, however, the guidance has been applied to all types of models, including those with no quantitative component or financial impact.
Furthermore, the Model Risk guidance also applies to what examiners call “non-models”: even if a process does not qualify as a model, banks are required to document why. Per the Model Risk guidance: “While outside the scope of this guidance, more qualitative approaches used by banking organizations— i.e., those not defined as models according to this guidance—should also be subject to a rigorous control process.” The resulting requirements for documentation and process have grown as a result.
General Effects
The impact of this “guidance” on banking is profound. Model validation functions — in the business, in compliance, in internal audit — have swelled. Process has swallowed innovation in many areas of bank operations.
Examiners insist on monitoring metrics for models that often are not risk based or in any way constructive. Examiners favor adding metrics because they are easy to audit, and any deviations can be flagged. This has led to large suites of metrics for many of the models. These additional metrics create little actionable information and create false positive breaches, in addition to inefficient model management. This creates confusion for the model risk committees that banks are required to form, who may be presented with numerous — unimportant — breaches in nearly every monitoring cycle.
Variable Application
Regulatory oversight on models is highly influenced by each individual examiner’s interpretation of the scope and substance of the guidance, as well as the examiner’s own view of the best methodological approach to developing a model. Banks may have legitimate differences of opinion in how to model a given risk or process — indeed, competition here is the essence of banking — but examiners may overrule that judgement. A “horizontal review” may result in examiners forcing some banks to adopt a process that they liked better at another bank; examiners from different agencies may have different preferences. For example, one bank reports that not monitoring specific cell-level accuracy for a risk rating model was deemed a significant gap by the OCC, but was never considered by the FRB and the FDIC. A recent BPI call among modeling teams included discussion of whether modeling practices needed to change because of “whispers” from a consulting firm that examiners were changing data lineage requirements for certain models.
It may be helpful to highlight how the guidance plays out in a few areas of bank operations.
Case Studies
AML
In 2021, FinCEN and the banking agencies sought comment on “the extent to which the principles discussed in the MRMG [Model Risk Management Guidance] support compliance by banks” with AML and OFAC requirements. A BPI comment letter explained the then status quo:
BPI members report widely divergent approaches in how they apply the MRMG in this context. Several institutions report that they consider all or almost all BSA/AML and many sanctions tools to be models under the MRMG. Others consider far fewer BSA/AML tools and no or almost no sanctions tools to be models. Member institutions also report differences in how they apply the MRMG to BSA/AML and sanctions tools that are determined to be models. Some institutions permit substantial tailoring of the MRMG elements for validation of BSA/AML and sanctions tools; some institutions have a process to grant waivers for certain MRMG-related procedures; and some institutions do not meaningfully differentiate how they apply the MRMG elements to models, whether they are BSA/AML or sanctions tools or other models. Member institutions also differ in the extent to which they permit BSA/AML and sanctions tools to be expedited into production.
The agencies subsequently issued an Interagency Statement on how the guidance relates to anti-money laundering; however, little change has occurred in the field.
The underlying problem is that the tools that institutions employ to monitor for BSA/AML compliance differ markedly from models for which the Model Risk guidance was designed. They often rely on subjective human review of outputs, such as alerts of potential suspicious activity or sanctions hits, and information on the utility or practical application of the ultimate product of these tools, especially suspicious activity reports, is generally unavailable. These and other differences can affect, even for those BSA/AML and sanctions tools determined to be models, how institutions undertake validation, and frequently render aspects of the MRMG inapplicable in whole or in part to validation of these tools.
Particularly with regard to AML models, which must be modified to adapt to changing circumstances, it is important for banks to be able to introduce a model change and operate it for a review period, as a pilot, but the guidance generally does not allow for it.
Artificial Intelligence
The Model Risk guidance was issued prior to the advent of Generative AI and hasn’t been updated for Gen AI. Again, the Model Risk guidance was designed for systems that produce a number, and can be backtested to see if that number was correct. It is a poor fit for generative AI that produces text, audio or video.
As a result, demonstrating compliance with the guidance can slow down the innovation in generative AI, especially the usage of Gen AI in efficiency, productivity and operation optimization. The regulation requires extensive documentation and comprehensive testing from model owners and model validators that could cover bias and explainability, which can delay the time to release into the production and the cost. For example, for Gen AI tools such as Code Whisperer, a firm would have to wait for a few months for the model to go through the development and validation cycles. In many cases, external firms don’t want to sell to banks, because they can’t make a profit if they have to go through the model validation process.
Banks again report extraordinary variation in whether their examiners can “get comfortable” with AI. Some understand that banks face a round peg/square hole problem; others believe that such a poor fit is a reason to prevent adoption. Almost all banks report that they are delaying justifiable and effective use of AI in some of their operations pending greater examiner “comfort.”
Sanctions
A common misunderstanding of sanctions compliance is that it is simply a matter of banks checking the names of current or prospective customers against the sanctions list maintained by the Office of Foreign Assets Control (OFAC) at the Treasury Department. The truth is that people or organizations on the sanctions list immediately cease transacting in their own name, and rather employ shell companies or nominees. Thus, sanctions compliance now consists of an intelligence gathering operation to identify those companies or nominees. Sanctions filters are not designed for predictive quantitative analysis or financial decision-making (the focus of the Model Risk guidance); nor do they predict or “estimate” potential sanctions activity. Instead, sanctions filters are designed to identify sanctions concerns within a given transaction, as that transaction occurs, and generate an alert for subsequent investigation. They also need to be adaptable in ways that the Model Risk guidance discourages.
The Model Risk guidance does not comprehend such concepts. The result is predictable. From an earlier comment letter:
BPI member institutions report that applying MRMG validation approaches to sanctions screening tools has pushed institutions to spend significant resources on documenting detection or non-detection of typographically, linguistically, and commercially unrealistic permutations of sanctioned party names (e.g., “N0Rth K0Re@”). As another example, BPI member institutions report that applying the MRMG, including in light of related examiner feedback, has made it difficult to turn off or discontinue tool scenarios or rules, even in circumstances in which the scenarios or rules have led to the filing of very few SARs.
As a result, banks report that a significant portion of their sanctions-related exam questions from the federal banking agencies can pertain to model validation for sanctions filters.
Cybersecurity and Anti-Fraud
Another more important use case would be cybersecurity and fraud. Criminals can employ the most advanced Gen AI tools to help commit various frauds or scams. Ideally, banks would be able to use the same tools to defend themselves, but compliance with the Model Risk guidance and its requirement for documentation and multiple levels of validation prevents them from fighting fire with fire. Even if these Gen AI anti-fraud techniques aren’t models under MRM, they can still be considered non-models and therefore expected to have costly and time-consuming validation regimes.
Conclusion
The fact that the Model Risk Management Guidance has for years been applied by the federal banking agencies in violation of law should be sufficient reason to rescind it. But its rescission would also be a substantial boost in the ability of banks to innovate and operate efficiently, with an intangible but sizeable boost to economic growth. And the fact that it is illegal means that it can and should be withdrawn immediately.
[1] https://public-inspection.federalregister.gov/2025-03138.pdf. That Executive Order defines regulations to include any “guidance document.”
