# Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics

arXiv: https://arxiv.org/abs/2608.16084 · HTML: https://mlsanity.com/paper/2608.16084/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Builds on (most cited first, 30 of 35)

- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Neural Operators for Accelerating Scientific Simulations and Design](https://mlsanity.com/paper/2309.15325/) (2023)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [Traditional and Heavy-Tailed Self Regularization in Neural Network Models](https://mlsanity.com/paper/1901.08276/) (2019)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [The Emergence of Spectral Universality in Deep Networks](https://mlsanity.com/paper/1802.09979/) (2018)
- [Hutch++: Optimal Stochastic Trace Estimation](https://mlsanity.com/paper/2010.09649/) (2020)
- [ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses](https://mlsanity.com/paper/2411.11268/) (2024)
- [Multi-stage Neural Networks: Function Approximator of Machine Precision](https://mlsanity.com/paper/2307.08934/) (2023)
- [Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks](https://mlsanity.com/paper/1901.08278/) (2019)
- [On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers](https://mlsanity.com/paper/2102.07346/) (2021)
- [Hessian Eigenspectra of More Realistic Nonlinear Models](https://mlsanity.com/paper/2103.01519/) (2021)
- [Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution](https://mlsanity.com/paper/2304.07029/) (2023)
- [LUCIE: A Lightweight Uncoupled ClImate Emulator with long-term stability and physical consistency for O(1000)-member ensembles](https://mlsanity.com/paper/2405.16297/) (2024)
- [Learning continuous models for continuous physics](https://mlsanity.com/paper/2202.08494/) (2022)
- [OceanNet: A principled neural operator-based digital twin for regional oceans](https://mlsanity.com/paper/2310.00813/) (2023)
- [Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos](https://mlsanity.com/paper/2503.18731/) (2025)
- [Multiplicative noise and heavy tails in stochastic optimization](https://mlsanity.com/paper/2006.06293/) (2020)
- [Tuning Frequency Bias of State Space Models](https://mlsanity.com/paper/2410.02035/) (2024)
- [On generalized residue network for deep learning of unknown dynamical systems](https://mlsanity.com/paper/2002.02528/) (2020)
- [Models of Heavy-Tailed Mechanistic Universality](https://mlsanity.com/paper/2506.03470/) (2025)
- [Hierarchical Implicit Neural Emulators](https://mlsanity.com/paper/2506.04528/) (2025)
- [Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence](https://mlsanity.com/paper/2205.04601/) (2022)
- [Simultaneous emulation and downscaling with physically-consistent deep learning-based regional ocean emulators](https://mlsanity.com/paper/2501.05058/) (2025)

## Abstract (arXiv)

Neural autoregressive models have rapidly emerged as powerful emulators of high-dimensional chaotic systems, yet their long-term instability and error growth remain poorly understood, leading to ad-hoc solutions. Here, we develop an eigenanalysis framework that reveals the dynamical origin of this error growth. By analyzing the Jacobian of the learned one-step update map with respect to the state, we show how inference-time error growth, and thus model stability, is governed by its spectral radius. Direct-step architectures (models that predict the next state from the previous one) generically admit unstable eigenvalues with magnitudes exceeding one, explaining the rapid divergence of these widely used models. In contrast, integration-constrained models (where the time derivative is estimated and integrated with a higher-order integrator) collapse their eigenspectrum onto the unit circle, yielding neutral stability and a universal linear error-scaling law. The largest eigenvalue of this Jacobian provides an architecture-agnostic, a priori diagnostic of short-term skill, long-term stability, and spectral bias, without requiring an expensive rollout. Leveraging this theory, we introduce a stability-promoting loss that explicitly regularizes Jacobian-driven error amplification, improving both forecast accuracy and dynamical robustness. Demonstrated across 29 models spanning two architectures, several explicit and implicit integrators, and multiple loss functions on the Kuramoto-Sivashinsky system, our results establish a theoretical foundation for the design and evaluation of neural emulators of chaotic multi-scale dynamics. More broadly, our framework is a step toward the kind of a priori stability analysis that numerical analysis provides for discretizations of differential equations and that scientific machine learning currently lacks.