In early November 2024, the Network for the Greening of the Financial System (NGFS), a consortium of central banks and other regulatory supervisors across the globe (notably excluding the Fed and other U.S. regulators, who exited early this year), announced that a new climate damage function had been incorporated into its climate scenario toolkit. Based on an academic article in Nature, the new climate damage function implies dramatically higher economic losses from worsening climate conditions: a 19% loss of global real income by 2050 and a 60% loss by 2100. These revised estimates of economic collapse are unprecedented in size. Livio Stracca, Chair of the NGFS workstream “Scenario and Design Analysis” and Deputy Director General Financial Stability at the European Central Bank (ECB), said that “the scenarios show that climate change is becoming a first order factor for our economies.” The very large economic damage opens the door to more binding bank transition targets, higher fines on banks for inadequate compliance with ECB climate change risk expectations and potentially additional bank capital requirements for climate risks.
In early December 2024, the BPI study “The NGFS’s New Climate Damage Function: A Flawed Analysis with Massive Economic Consequences” identified serious statistical issues with the new damage function.[1] Sometime after that, Nature put up a warning on the academic paper, saying “Readers are alerted that the reliability of data and methodology presented in this manuscript is currently in question. Appropriate editorial action will be taken once this matter is resolved.”[2] The BPI study was subsequently sent to the corresponding author of the Nature paper and to the editors at Nature for consideration and comment but received no response. Despite the academic warning from Nature, the NGFS did not remove the damage function from its climate scenarios or issue a warning to its climate scenario users that the data and methodology of the damage function were under review.
In August of this year, Nature published a comment paper it received in September 2024 that proved that a data error in the Nature paper had incorrectly inflated the economic loss numbers of the damage function by about a factor of three. The authors of the Nature paper responded by posting a revision in which they fixed the data error and then changed the model and data, significantly raising the numbers to roughly where they were in the original paper. Despite these revelations and the fact that the academic warning is still up on the Nature site as of this writing, the NGFS has taken no action. Worse, although revealed to the public a year later, the data error was known to have existed at least six weeks before the NGFS officially unveiled the new damage function. Why was the new damage function rolled out anyway in early November? And given that it was rolled out, why was it not retracted after Nature put up the warning?
Somehow, the NGFS process to put into production an important model failed to uncover an easily discoverable, but very serious, problem. What other serious problems may be lurking in NGFS scenarios? The NGFS needs to determine how the process failed and correct it if it is to restore confidence in its climate scenarios. Even though the NGFS failed to prevent a flawed model from going into production, the NGFS had another chance to act after the Nature warning was put up. That it took no action raises troubling questions about the objectivity of its member central banks and regulators in their climate risk supervision of banks.
The NGFS’s failure to act cannot be justified on the grounds that the revised model produces essentially the same damage estimates as the original model that contained the data error. The revised Nature model rests on the same flimsy statistical foundations as the original model. More importantly, a “start with the conclusion, then find the model that fits it” response to a damning data error should never be acceptable in bank regulatory policy. Regardless of the editorial action Nature ultimately decides to take, the NGFS should remove the damage function from its climate scenarios toolkit immediately.
Importance of NGFS Scenarios for Climate Regulation of Banks
It is essential to recognize that the NGFS is not just another international organization or NGO. The NGFS is a forum for the world’s central banks and other regulators to agree on a common set of climate risk principles, even if they disagree with others, and also a workstream to develop a common climate toolkit that can be used for bank supervision of the climate risks of banks. Central banks and other regulators, including some of the world’s most important regulators, rely on NGFS output. Essentially, central banks and other regulators create the climate methodologies they use for bank supervision, but they present them through the NGFS.
For example, in “ECB report on good practices for climate stress testing,” the ECB set out its climate risk expectations that banks must meet. [3] Banks that are insufficiently compliant in the ECB’s judgment can be fined as much as 5% of their daily turnover, assessed every day the infringement continues. In the report, the ECB endorsed the use of climate scenarios to meet regulatory expectations, noting that the most used scenarios are created by the NGFS, of which the ECB is an active member. In July of this year, in “Interactions between Climate Scenario Analysis and Transition Plans,” the NGFS recommended that banks use its climate scenarios to aid in climate transition risk planning as well as in climate target setting, a regulatory expectation and goal of the ECB. [4] Put another way, central banks and other regulators are advising banks to use the climate scenarios and toolkit that the central banks and regulators have created.
Similarly, in January 2025, the Financial Stability Board published “Assessment of Climate-related vulnerabilities, Analytical Framework and Toolkit.”[5] The note reports the new NGFS damage function as the baseline outlook for the economy if there are no climate policy changes.[6]
NGFS scenarios are therefore essential for bank compliance with regulatory climate risk expectations. The models that underlie them, and there are many besides climate damage functions, must be sound.
Background on Damage Function Timeline
The NGFS incorporated a new climate damage function — a prediction of the extent to which climate change reduces economic growth — into its scenario toolkit last year. It relied on published academic research in Nature in April 2024, Kotz et al.[7] Because it found startlingly high effects of climate change on the economy, the paper attracted substantial media attention. According to CarbonBrief, it was the second most cited climate paper in 2024.[8]
An independent test and validation of the Kotz et al. model were provided by a BPI study about a month after the NGFS model was rolled out, in early December 2024. The study re-estimated and re-simulated the Kotz et al. model under different assumptions, identifying some serious statistical issues with the model.
Sometime in late December or early January, Nature put up a warning on the Kotz et al. paper, saying “Readers are alerted that the reliability of data and methodology presented in this manuscript is currently in question. Appropriate editorial action will be taken once this matter is resolved.” However, Nature gave no indication of what the potential problems were, what was being considered, or how long the review would last.
On Aug 6, 2025, eight months later, Nature posted a “Matters Arising” by Bearpark et al.[9] Puzzled by the very large economic losses in the model, Bearpark et al. removed one country at a time from the Kotz et al. data set and re-simulated the model. They found a large mistake in the data for a relatively unimportant country, Uzbekistan. When they removed Uzbekistan from the data and re-simulated the Kotz et al. model, they found that the mistake in the data for a tiny country had inflated the estimates by a factor of about three. As one of the authors of Bearpark et al., Solomon Hsiang, told phys.org:
When we dropped Uzbekistan, suddenly everything changed. And we were like, ‘whoa, that’s not supposed to happen. We felt like we had to document it in this form because it’s been used so widely in policy making.[10]
Bearpark et al. also pointed out that Kotz et al. had underestimated the uncertainty of the estimates, which they showed under reasonable assumptions was substantially higher.
Kotz et al. responded with a revised paper that made a specification change to the regression model, restoring their initial estimates after the Uzbekistan data was corrected. [11] They also changed the model simulation, resulting in uncertainty estimates that were very similar to their initial paper’s results. Commenting on the model change, one of the authors of the study, Maximilian Kotz, told the Washington Post that “We find actually that by doing that, our estimates in general get more robust.”[12] The corresponding author, Leonie Wenz, told the Washington Post, “We are grateful, and I think it’s a good part of the scientific process that they’ve pointed out these issues. But importantly, the main conclusions of the paper hold, and there are only slight changes to the estimates.” [13] The Potsdam Institute for Climate Impact Research, the academic institution in which Kotz et al. are members, supported the authors by saying:
In response to feedback from other scientists, the authors of the paper “The economic commitment of climate change” at the Potsdam Institute for Climate Impact Research (PIK) have revised their analysis and are making it open access for the wider scientific community to engage with. PIK welcomes the critical scrutiny published as a “Matters Arising” in Nature as an important part of the scientific debate, and is committed to continuing to uphold the highest standards of research integrity and transparency.[14]
The statement went on to emphasize that the estimates were little changed after the revision. Meanwhile, the academic warning is still up at Nature as of this writing.
CBO Analysis for Congress is Corrupted
In December 2024, the Congressional Budget Office presented Congress with a study of how climate change will affect future GDP.[15] The study reported that GDP would be 4 percent lower in 2100 under current policies because of climate change. The study based its estimate on a study-of-studies. An accompanying working paper reported that Kotz et al. was one of the 15 studies used.[16] It showed how extreme the paper’s result is compared with other studies and noted that the paper by itself accounted for one-fourth of the projected decline in GDP.
Process Failure at the NGFS
Bearpark et al. was received by Nature on September 16, 2024, implying that the latest Nature and Kotz et al. became aware of a potentially serious flaw in the model was mid-September 2024, well before the NGFS officially announced the damage function in early November 2024. Most likely, they would have been aware earlier, since the typical process is for a critic to raise issues with the authors first before making a formal submission to Nature. The NGFS worked closely with the Nature paper modeling team and the Potsdam Institute to implement the damage function,[17] and so it is puzzling why the NGFS would have been unaware of it.
The data problem could have been confirmed quickly, in about a day or so, depending on computer speed. Verification requires a small code change to remove Uzbekistan from the regression dataset and then a re-estimation and re-simulation of the models.
In fact, the data problem could have been confirmed much more quickly than that, in just a few minutes. The BPI study showed that almost all the model’s economic damage results from a subset of the coefficients on the change in temperature times the level of temperature terms. It just takes a few minutes to remove Uzbekistan, rerun the regressions and confirm that the magnitudes of the coefficients drop quite a bit, implying significantly lower damage.
If, in addition to that simple test, we want an estimate of the effect of dropping Uzbekistan, we can use the results in the BPI study. It is possible to approximate the effect of the Kotz et al. model by estimating model regressions with eight, nine or ten temperature terms, and then averaging only the coefficients on the change in temperature multiplied by the level of temperature terms. Then, we can calculate the effect on global real income using a representative RCP 8.5 scenario using only the subset of the averaged terms. Figure 1 shows the result of that analysis, which is relatively quick to run.
Figure 1
As is apparent, the Kotz et al. model estimates drop very significantly without Uzbekistan.
Nature and Kotz et al. would have known about this problem on or before September 16, 2024 and it would have taken at most a day or two to verify the problem, and much more quickly than that if the results in the BPI study were used. We do not know if Nature or Kotz et al. informed the NGFS of the problem, but given that the model was being incorporated into the NGFS scenarios at that time, they certainly should have.
However, we do know that the Nature warning on the model was publicly posted on the article website and the NGFS did nothing in response. Shouldn’t the central bank members of the NGFS have asked Nature or Kotz et al. what the problem was? Shouldn’t the NGFS have retracted the model if they learned of the data error?
If the question “what is the problem with the Kotz et al. study” was asked but not answered for whatever reason, should regulators incorporate models for regulatory supervision that they cannot vet if there turns out to be a problem? If the question was not asked, then how can we be sure that regulators are objectively choosing the best research for supervisory purposes rather than pre-selecting conclusions and finding the evidence to back them up? Why would the NGFS decide to base its damage function on research whose effects are much larger than reported in most other academic studies?
A Quick Review of The Revised Nature Model
Bearpark et al. showed that the inclusion of the mistaken data from Uzbekistan inflated the economic losses by about a factor of three, implying that the NGFS damage function is very substantially overstated. Kotz et al. responded by adding an additional, and it turns out, very powerful term[18] to their regression model, which has the effect of restoring their original large economic losses. Kotz et al. justify the inclusion of the extra term by pointing out that other academic papers include it, and also that Simon Hsiang, one of the authors of Bearpark et al., advocated for its use in a blog post in 2019.
The problem with that justification is that if it is true now, it is equally true of the previous version of the model. Using the same methods from the BPI study as employed in Figure 1, we can estimate what would have happened had the extra term been included in the original model with the Uzbekistan data error. As Figure 2 suggests, the estimated losses are much too large to be even remotely credible.
Figure 2
Changing to a specification of the model in response to a damning data error when that specification should have equally been included in the previous version, but was not, should engender serious skepticism about the methodology and results. Commenting in the Washington Post, Simon Hsiang said: “Science doesn’t work by changing the setup of an experiment to get the answer you want. This approach is antithetical to science.”[19]
Kotz et al. responded to another criticism from Bearpark et al. by modifying the way in which they simulate the model. The change does not really respond to Bearpark et al.’s point, but it does largely restore the Kotz et al.’s original uncertainty estimates, suggesting again a specification change to achieve a targeted conclusion.[20]
Besides that, the same statistical problems noted in the BPI study remain in the revised model. For example, when including the extra term in the regression and excluding Uzbekistan from the data, none of the coefficients that produce almost all the model’s losses are statistically significant if correlation across time is accounted for.[21] Other papers on climate damage functions account for correlation across time.
Conclusion
In November 2024, central banks and other regulators across the globe, under the auspices of the NGFS, put into their climate toolkit a climate damage function that could justify more stringent climate targets for banks, fines for banks that operate in Europe and potentially additional capital. About a month after the new damage function was released, a BPI study showed that the climate damage function had some serious statistical problems. It should not have been terribly surprising then to learn the climate damage function also had a serious data error that inflated its loss numbers by about a factor of three.
What is surprising is the reaction of the NGFS. The regulators in the NGFS could have, and should have, known about the data error before putting the new model into their climate scenario toolkit. They also could have investigated and acted when the academic paper the climate damage function was based on was called into question by the academic journal that published it. But they took no action and still take no action.
The revised climate damage model is even more flawed than the original, since the statistical problems remain and it now appears that the model update was cherry-picked to reach a pre-determined conclusion. The NGFS should immediately retract the damage model from its climate scenario toolkit. The NGFS must also review and correct its procedures for putting its climate models into production to prevent a model with a serious data error from again being adopted, especially when the error should have been caught. A general review and update of NGFS model control procedures is necessary to restore faith in their climate toolkit and data.
The more troubling questions remain unanswered: did any central bank or other regulator follow up with Nature or Kotz et al. when it was publicly revealed that there was a problem with the model they use to regulate banks? If they did not, why not? Are regulators putting their thumb on the scale, targeting a pre-determined conclusion rather than using the best scientific evidence?
[1] Hopper, G, “The NGFS’s New Climate Damage Function: A Flawed Analysis With Massive Economic Consequences,” Bank Policy Institute, available at https://bpi.com/the-ngfss-new-climate-damage-function-a-flawed-analysis-with-massive-economic-consequences/
[2] The warning, which was dated November 6, 2024, and still present on the Nature website at the time of this writing, was put up sometime after late December 2024, as can be verified with the Wayback Machine.
[3] ECB, “ECB report on good practices for climate stress testing,” (2022), available at https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.202212_ECBreport_on_good_practices_for_CST~539227e0c1.en.pdf
[4] NGFS, “Interactions between Climate Scenario Analysis and Transition Plans,” (2025), available at https://www.ngfs.net/system/files/2025-07/NGFS_Interactions%20between%20climate%20scenario%20analysis%20and%20Transition%20plans.pdf
[5] https://www.fsb.org/uploads/P160125.pdf
[6] See Graph 3, page 17.
[7] Kotz, M, Leverman, A and Wenz, L, “The Economic Commitment of Climate Change,” Nature (2024), available at https://www.nature.com/articles/s41586-024-07219-0
[8] See https://www.carbonbrief.org/analysis-the-climate-papers-most-featured-in-the-media-in-2024/
[9] Bearpark, T, Hogan, D and Hsiang, S, “Data Anomalies and the Economic Commitment of Climate Change,” Nature, (2025) available at https://www.nature.com/articles/s41586-025-09320-4
[10] See https://phys.org/news/2025-08-major-climate-gdp.html
[11] Kotz et al., “Author correction of ‘The economic commitment of climate change’”, 2025, available at https://zenodo.org/records/15984134
[12] https://www.washingtonpost.com/climate-environment/2025/08/06/nature-study-flawed-climate-damages/
[13] https://www.washingtonpost.com/climate-environment/2025/08/06/nature-study-flawed-climate-damages/
[14] See statement from PIK, available at https://www.pik-potsdam.de/en/news/latest-news/nature-study-on-economic-damages-from-climate-change-revised
[15] https://www.cbo.gov/system/files/2024-12/60845-climate-risk.pdf
[16] https://www.cbo.gov/system/files/2025-02/61186-Climate-GDP.pdf
[17] See the presentation on the damage function at the launch event of the NGFS scenarios Phase V, available at https://www.youtube.com/watch?v=oII1qey4T_Y
[18] Technically speaking, Kotz et al. added a quadratic time trend in addition to the linear time trend they already had.
[19] https://www.washingtonpost.com/climate-environment/2025/08/06/nature-study-flawed-climate-damages/
[20] Kotz et al. switch to Conley standard errors, which adjust the standard errors in regression models for spatial correlations. Kotz et al. simulate their revised model by drawing from the covariance matrix of the regression estimates, calculated with Conley standard errors to address Bearpark et al.’s criticism that the previous model simulation had underestimated uncertainty. Although Kotz et al.’s simulation methodology is sometimes performed, it should be noted that the covariance matrix of the regression estimates does not represent the true probability distribution of the parameters, i.e., the posterior distribution. Even if it did reflect the true distribution, it is only asymptotically normal: it is not normal in finite samples. Bearpark et al. make a sensible and plausible point that Kotz et al. do not effectively answer.
[21] Statistical significance of the most important coefficients under the revised model vanishes at conventional significance levels if we double cluster the standard errors by region and year.
