# Improved deep learning of chaotic dynamical systems with multistep penalty losses

arXiv: https://arxiv.org/abs/2410.05572 · HTML: https://mlsanity.com/paper/2410.05572/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?](https://mlsanity.com/paper/1801.03744/) (2018)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [LUCIE: A Lightweight Uncoupled ClImate Emulator with long-term stability and physical consistency for O(1000)-member ensembles](https://mlsanity.com/paper/2405.16297/) (2024)
- [OceanNet: A principled neural operator-based digital twin for regional oceans](https://mlsanity.com/paper/2310.00813/) (2023)
- [Differentiable physics-enabled closure modeling for Burgers' turbulence](https://mlsanity.com/paper/2209.11614/) (2022)

## Built on by (most cited first, 3 of 3)

- [Binned Spectral Power Loss for Improved Prediction of Chaotic Systems](https://mlsanity.com/paper/2502.00472/) (2025)
- [A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series](https://mlsanity.com/paper/2511.06609/) (2025)
- [Staying on the Attractor: Supervising Neural Surrogates of 3D Turbulence Where They Leave It](https://mlsanity.com/paper/2609.32864/) (2026)

## Abstract (arXiv)

Predicting the long-term behavior of chaotic systems remains a formidable challenge due to their extreme sensitivity to initial conditions and the inherent limitations of traditional data-driven modeling approaches. This paper introduces a novel framework that addresses these challenges by leveraging the recently proposed multi-step penalty (MP) optimization technique. Our approach extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs. By introducing penalized local discontinuities in the forecast trajectory, we effectively handle the non-convexity of loss landscapes commonly encountered in training neural networks for chaotic systems. We demonstrate the effectiveness of our method through its application to two challenging use-cases: the prediction of flow velocity evolution in two-dimensional turbulence and ocean dynamics using reanalysis data. Our results highlight the potential of this approach for accurate and stable long-term prediction of chaotic dynamics, paving the way for new advancements in data-driven modeling of complex natural phenomena.