# Numerical models outperform AI weather forecasts of record-breaking extremes

arXiv: https://arxiv.org/abs/2508.15724 · HTML: https://mlsanity.com/paper/2508.15724/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Learning in High Dimension Always Amounts to Extrapolation](https://mlsanity.com/paper/2110.09485/) (2021)
- [ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses](https://mlsanity.com/paper/2411.11268/) (2024)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [Can AI weather models predict out-of-distribution gray swan tropical cyclones?](https://mlsanity.com/paper/2410.14932/) (2024)
- [Forecaster's Dilemma: Extreme Events and Forecast Evaluation](https://mlsanity.com/paper/1512.09244/) (2015)
- [Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events](https://mlsanity.com/paper/2404.17652/) (2024)
- [Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciar'an](https://mlsanity.com/paper/2312.02658/) (2023)
- [Evaluation of Tropical Cyclone Track and Intensity Forecasts from Artificial Intelligence Weather Prediction (AIWP) Models](https://mlsanity.com/paper/2409.06735/) (2024)
- [Progression: an extrapolation principle for regression](https://mlsanity.com/paper/2410.23246/) (2024)
- [Constructing Extreme Heatwave Storylines with Differentiable Climate Models](https://mlsanity.com/paper/2506.10660/) (2025)
- [Deep Learning Atmospheric Models Reliably Simulate Out-of-Sample Land Heat and Cold Wave Frequencies](https://mlsanity.com/paper/2507.03176/) (2025)

## Built on by (most cited first, 14 of 14)

- [Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical features](https://mlsanity.com/paper/2512.24440/) (2025)
- [Learning to Advect: A Neural Semi-Lagrangian Architecture for Weather Forecasting](https://mlsanity.com/paper/2601.21151/) (2026)
- [AI-boosted rare event sampling to characterize extreme weather](https://mlsanity.com/paper/2510.27066/) (2025)
- [Forecasting the Future with Yesterday's Climate: Temperature Bias in AI Weather and Climate Models](https://mlsanity.com/paper/2509.22359/) (2025)
- [Benchmarking atmospheric circulation variability in an AI emulator, ACE2, and a hybrid model, NeuralGCM](https://mlsanity.com/paper/2510.04466/) (2025)
- [On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification](https://mlsanity.com/paper/2511.17176/) (2025)
- [Balancing Accuracy and Speed: A Multi-Fidelity Ensemble Kalman Filter with a Machine Learning Surrogate Model](https://mlsanity.com/paper/2512.12276/) (2025)
- [Learning vertical coordinates via automatic differentiation of a dynamical core](https://mlsanity.com/paper/2512.17877/) (2025)
- [Extrapolation in Statistical Learning with Extreme Value Theory](https://mlsanity.com/paper/2605.01909/) (2026)
- [Assessing Extrapolation of Peaks Over Thresholds with Martingale Testing](https://mlsanity.com/paper/2512.03116/) (2025)
- [Skillful Subseasonal-to-Seasonal Forecasting of Extreme Events with a Multi-Sphere Coupled Probabilistic Model](https://mlsanity.com/paper/2512.12545/) (2025)
- [Integrating GNSS-Derived Zenith Wet Delay into a Weather Foundation Model Improves Precipitation Forecasting](https://mlsanity.com/paper/2607.05658/) (2026)
- [AI Emulation of Stochastic Sudden Stratospheric Warming with Interpretable Latent Structure](https://mlsanity.com/paper/2610.02069/) (2026)
- [Artificial intelligence pathways from weather to climate](https://mlsanity.com/paper/2610.09770/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI)-based models are revolutionizing weather forecasting and have surpassed leading numerical weather prediction systems on various benchmark tasks. However, their ability to extrapolate and reliably forecast unprecedented extreme events remains unclear. Here, we show that for record-breaking weather extremes, the numerical model High RESolution forecast (HRES) from the European Centre for Medium-Range Weather Forecasts still consistently outperforms state-of-the-art AI models GraphCast, GraphCast operational, Pangu-Weather, Pangu-Weather operational, and Fuxi. We demonstrate that forecast errors in AI models are consistently larger for record-breaking heat, cold, and wind than in HRES across nearly all lead times. We further find that the examined AI models tend to underestimate both the frequency and intensity of record-breaking events, and they underpredict hot records and overestimate cold records with growing errors for larger record exceedance. Our findings underscore the current limitations of AI weather models in extrapolating beyond their training domain and in forecasting the potentially most impactful record-breaking weather events that are particularly frequent in a rapidly warming climate. Further rigorous verification and model development is needed before these models can be solely relied upon for high-stakes applications such as early warning systems and disaster management.