# Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks

arXiv: https://arxiv.org/abs/2304.12865 · HTML: https://mlsanity.com/paper/2304.12865/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 67 references that are in the map.
- First posted on arXiv 24 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [AI Poincar'e: Machine Learning Conservation Laws from Trajectories](https://mlsanity.com/paper/2011.04698/) (2020)
- [Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers](https://mlsanity.com/paper/1910.00659/) (2019)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [Noether Networks: Meta-Learning Useful Conserved Quantities](https://mlsanity.com/paper/2112.03321/) (2021)
- [Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation](https://mlsanity.com/paper/2109.12269/) (2021)
- [Physics-Informed Echo State Networks](https://mlsanity.com/paper/2011.02280/) (2020)
- [Towards Physically-consistent, Data-driven Models of Convection](https://mlsanity.com/paper/2002.08525/) (2020)
- [Automatic-differentiated Physics-Informed Echo State Network (API-ESN)](https://mlsanity.com/paper/2101.00002/) (2020)
- [Discovering conservation laws from trajectories via machine learning](https://mlsanity.com/paper/2102.04008/) (2021)

## Built on by (most cited first, 6 of 6)

- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [A probabilistic framework for learning non-intrusive corrections to long-time climate simulations from short-time training data](https://mlsanity.com/paper/2408.02688/) (2024)
- [Invariant Measures in Time-Delay Coordinates for Unique Dynamical System Identification](https://mlsanity.com/paper/2412.00589/) (2024)
- [Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence](https://mlsanity.com/paper/2305.00100/) (2023)
- [Detecting Invariant Manifolds in ReLU-Based RNNs](https://mlsanity.com/paper/2510.03814/) (2025)
- [Calibrating Geophysical Predictions under Constrained Probabilistic Distributions](https://mlsanity.com/paper/2512.03081/) (2025)

## Abstract (arXiv)

Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants--such as the Lyapunov exponent spectrum and fractal dimension--in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using the recurrent neural network architecture of reservoir computing. Results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model, both typical test cases for numerical weather prediction.