# Recommendations for Comprehensive and Independent Evaluation of Machine Learning-Based Earth System Models

arXiv: https://arxiv.org/abs/2410.19882 · HTML: https://mlsanity.com/paper/2410.19882/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 2 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Built on by (most cited first, 6 of 6)

- [Deep Learning Atmospheric Models Reliably Simulate Out-of-Sample Land Heat and Cold Wave Frequencies](https://mlsanity.com/paper/2507.03176/) (2025)
- [Benchmarking atmospheric circulation variability in an AI emulator, ACE2, and a hybrid model, NeuralGCM](https://mlsanity.com/paper/2510.04466/) (2025)
- [Hierarchical Testing of a Hybrid Machine Learning-Physics Global Atmosphere Model](https://mlsanity.com/paper/2602.11313/) (2026)
- [How Can Machine Learning Emulators Best Support Climate Science?](https://mlsanity.com/paper/2603.22320/) (2026)
- [Strategic Governance of AI Models in Earth Science](https://mlsanity.com/paper/2610.10560/) (2026)
- [FireGraphNet: Sub-daily wildfire spread prediction with directed graph neural networks](https://mlsanity.com/paper/2610.19191/) (2026)

## Abstract (arXiv)

Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics-based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop forecasting models into Earth-system models (ESMs), capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes. Modeling the Earth system is a much more difficult problem than weather forecasting, not least because the model must represent the alternate (e.g., future) coupled states of the system for which there are no historical observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML-based models, demonstrating the credibility of ML-based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML-based ESMs to strengthen their credibility and promote their wider use.