# Benchmarking atmospheric circulation variability in an AI emulator, ACE2, and a hybrid model, NeuralGCM

arXiv: https://arxiv.org/abs/2510.04466 · HTML: https://mlsanity.com/paper/2510.04466/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 75 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses](https://mlsanity.com/paper/2411.11268/) (2024)
- [Samudra: An AI Global Ocean Emulator for Climate](https://mlsanity.com/paper/2412.03795/) (2024)
- [Can AI weather models predict out-of-distribution gray swan tropical cyclones?](https://mlsanity.com/paper/2410.14932/) (2024)
- [Dynamical Tests of a Deep-Learning Weather Prediction Model](https://mlsanity.com/paper/2309.10867/) (2023)
- [Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate science](https://mlsanity.com/paper/2005.11862/) (2020)
- [Numerical models outperform AI weather forecasts of record-breaking extremes](https://mlsanity.com/paper/2508.15724/) (2025)
- [Recommendations for Comprehensive and Independent Evaluation of Machine Learning-Based Earth System Models](https://mlsanity.com/paper/2410.19882/) (2024)
- [Deep Learning Atmospheric Models Reliably Simulate Out-of-Sample Land Heat and Cold Wave Frequencies](https://mlsanity.com/paper/2507.03176/) (2025)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Testing of a Hybrid Machine Learning-Physics Global Atmosphere Model](https://mlsanity.com/paper/2602.11313/) (2026)

## Abstract (arXiv)

Physics-based atmosphere-land models with prescribed sea surface temperature have notable successes but also biases in their ability to represent atmospheric variability compared to observations. Recently, AI emulators and hybrid models have emerged with the potential to overcome these biases, but still require systematic evaluation against metrics grounded in fundamental atmospheric dynamics. Here, we evaluate the representation of four atmospheric variability benchmarking metrics in a fully data-driven AI emulator (ACE2-ERA5) and hybrid model (NeuralGCM). The hybrid model and emulator can capture the spectra of large-scale tropical waves and extratropical eddy-mean flow interactions, including critical levels. However, both struggle to capture the timescales associated with quasi-biennial oscillation (QBO, ~ 28 months) and Southern annular mode propagation (~ 150 days). These dynamical metrics serve as an initial benchmarking tool to inform AI model development and understand their limitations, which may be essential for out-of-distribution applications (e.g., extrapolating to unseen climates).