# Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution

arXiv: https://arxiv.org/abs/2304.07029 · HTML: https://mlsanity.com/paper/2304.07029/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2023.

## Built on by (most cited first, 30 of 31)

- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [OceanNet: A principled neural operator-based digital twin for regional oceans](https://mlsanity.com/paper/2310.00813/) (2023)
- [Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos](https://mlsanity.com/paper/2503.18731/) (2025)
- [Swift: An Autoregressive Consistency Model for Efficient Weather Forecasting](https://mlsanity.com/paper/2509.25631/) (2025)
- [Weather Prediction with Diffusion Guided by Realistic Forecast Processes](https://mlsanity.com/paper/2402.06666/) (2024)
- [Hierarchical Implicit Neural Emulators](https://mlsanity.com/paper/2506.04528/) (2025)
- [AERIS: Argonne Earth Systems Model for Reliable and Skillful Predictions](https://mlsanity.com/paper/2509.13523/) (2025)
- [The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators](https://mlsanity.com/paper/2510.06646/) (2025)
- [What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning](https://mlsanity.com/paper/2411.15101/) (2024)
- [Causal Climate Emulation with Bayesian Filtering](https://mlsanity.com/paper/2506.09891/) (2025)
- [Lazy Diffusion: Mitigating spectral collapse in generative diffusion-based stable autoregressive emulation of turbulent flows](https://mlsanity.com/paper/2512.09572/) (2025)
- [Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts](https://mlsanity.com/paper/2605.30184/) (2026)
- [Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence](https://mlsanity.com/paper/2305.00100/) (2023)
- [Spectrum-Informed Multistage Neural Networks: Multiscale Function Approximators of Machine Precision](https://mlsanity.com/paper/2407.17213/) (2024)
- [AI-boosted rare event sampling to characterize extreme weather](https://mlsanity.com/paper/2510.27066/) (2025)
- [WIND: Weather Inverse Diffusion for Zero-Shot Atmospheric Modeling](https://mlsanity.com/paper/2602.03924/) (2026)
- [On instabilities in neural network-based physics simulators](https://mlsanity.com/paper/2406.13101/) (2024)
- [Deep Learning Weather Models for Subregional Ocean Forecasting: A Case Study on the Canary Current Upwelling System](https://mlsanity.com/paper/2505.24429/) (2025)
- [Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence](https://mlsanity.com/paper/2504.15487/) (2025)
- [NORi: An ML-Augmented Ocean Boundary Layer Parameterization](https://mlsanity.com/paper/2512.04452/) (2025)
- [Towards Infinitely Long Neural Simulations: Self-Refining Neural Surrogate Models for Dynamical Systems](https://mlsanity.com/paper/2603.17750/) (2026)
- [Learning to Emulate Chaos: Adversarial Optimal Transport Regularization](https://mlsanity.com/paper/2604.21097/) (2026)
- [Samudra 2: Scaling Ocean Emulators across Resolutions](https://mlsanity.com/paper/2606.02610/) (2026)
- [Generating time-consistent dynamics with discriminator-guided image diffusion models](https://mlsanity.com/paper/2505.09089/) (2025)
- [Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics](https://mlsanity.com/paper/2608.16084/) (2026)
- [Missing the Butterfly and Predicting the Past: Features or Bugs of Accurate AI Weather Models?](https://mlsanity.com/paper/2608.25835/) (2026)
- [Aries: A Proprietary Medium-Range Weather Prediction Model for the Energy Industry](https://mlsanity.com/paper/2609.13292/) (2026)
- [Learning Hierarchical Causal Representations of the Effects of Forcings on Temperature in Climate Models](https://mlsanity.com/paper/2609.30995/) (2026)
- [CasEm: A Cascade Architecture for Long-Horizon Neural Emulation](https://mlsanity.com/paper/2609.34246/) (2026)
- [Counterfactual Predictions in Scientific Emulators Without Controlled Experiments](https://mlsanity.com/paper/2610.02252/) (2026)

## Abstract (arXiv)

Long-term stability and physical consistency are critical properties for AI-based weather models if they are going to be used for subseasonal-to-seasonal forecasts or beyond, e.g., climate change projection. However, current AI-based weather models can only provide short-term forecasts accurately since they become unstable or physically inconsistent when time-integrated beyond a few weeks or a few months. Either they exhibit numerical blow-up or hallucinate unrealistic dynamics of the atmospheric variables, akin to the current class of autoregressive large language models. The cause of the instabilities is unknown, and the methods that are used to improve their stability horizons are ad-hoc and lack rigorous theory. In this paper, we reveal that the universal causal mechanism for these instabilities in any turbulent flow is due to textitspectral bias wherein, textitany deep learning architecture is biased to learn only the large-scale dynamics and ignores the small scales completely. We further elucidate how turbulence physics and the absence of convergence in deep learning-based time-integrators amplify this bias, leading to unstable error propagation. Finally, using the quasi-geostrophic flow and European Center for Medium-Range Weather Forecasting (ECMWF) Reanalysis data as test cases, we bridge the gap between deep learning theory and numerical analysis to propose one mitigative solution to such unphysical behavior. We develop long-term physically-consistent data-driven models for the climate system and demonstrate accurate short-term forecasts, and hundreds of years of time-integration with accurate mean and variability.