# Model Reduction with Memory and the Machine Learning of Dynamical Systems

arXiv: https://arxiv.org/abs/1808.04258 · HTML: https://mlsanity.com/paper/1808.04258/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2018.

## Built on by (most cited first, 21 of 21)

- [Overview frequency principle/spectral bias in deep learning](https://mlsanity.com/paper/2201.07395/) (2022)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [On the Benefits of Memory for Modeling Time-Dependent PDEs](https://mlsanity.com/paper/2409.02313/) (2024)
- [Learning About Structural Errors in Models of Complex Dynamical Systems](https://mlsanity.com/paper/2401.00035/) (2023)
- [Physics-aware, probabilistic model order reduction with guaranteed stability](https://mlsanity.com/paper/2101.05834/) (2021)
- [Data-Driven Stochastic Closure Modeling via Conditional Diffusion Model and Neural Operator](https://mlsanity.com/paper/2408.02965/) (2024)
- [Thermodynamics of learning physical phenomena](https://mlsanity.com/paper/2207.12749/) (2022)
- [Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case](https://mlsanity.com/paper/2110.00546/) (2021)
- [Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Ito Diffusions](https://mlsanity.com/paper/2105.10102/) (2021)
- [Data-driven Effective Modeling of Multiscale Stochastic Dynamical Systems](https://mlsanity.com/paper/2408.14821/) (2024)
- [A comparison of Single- and Double-generator formalisms for Thermodynamics-Informed Neural Networks](https://mlsanity.com/paper/2404.01060/) (2024)
- [Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs](https://mlsanity.com/paper/2202.12373/) (2022)
- [Regression-based projection for learning Mori-Zwanzig operators](https://mlsanity.com/paper/2205.05135/) (2022)
- [Learning Epidemiological Dynamics via the Finite Expression Method](https://mlsanity.com/paper/2412.21049/) (2024)
- [Towards Model Reduction for Power System Transients with Physics-Informed PDE](https://mlsanity.com/paper/2110.14066/) (2021)
- [Can Transformers overcome the lack of data in the simulation of history-dependent flows?](https://mlsanity.com/paper/2512.16305/) (2025)
- [Deep-layer limit and stability analysis of the basic forward-backward-splitting induced network (II): learning problems](https://mlsanity.com/paper/2605.27133/) (2026)
- [Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs](https://mlsanity.com/paper/2608.18148/) (2026)
- [Memory-Conditioned Diffusion Model for Generalized Langevin Dynamics](https://mlsanity.com/paper/2609.28371/) (2026)
- [Neural Fields Encode Adaptation Geometry](https://mlsanity.com/paper/2610.07253/) (2026)
- [Spatial Context or Memory? Sizing the Inputs of Neural Surrogates for Partially Observed Fields](https://mlsanity.com/paper/2610.12723/) (2026)

## Abstract (arXiv)

The well-known Mori-Zwanzig theory tells us that model reduction leads to memory effect. For a long time, modeling the memory effect accurately and efficiently has been an important but nearly impossible task in developing a good reduced model. In this work, we explore a natural analogy between recurrent neural networks and the Mori-Zwanzig formalism to establish a systematic approach for developing reduced models with memory. Two training models-a direct training model and a dynamically coupled training model-are proposed and compared. We apply these methods to the Kuramoto-Sivashinsky equation and the Navier-Stokes equation. Numerical experiments show that the proposed method can produce reduced model with good performance on both short-term prediction and long-term statistical properties.