# Recurrent Neural Networks for Dynamical Systems: Applications to Ordinary Differential Equations, Collective Motion, and Hydrological Modeling

arXiv: https://arxiv.org/abs/2202.07022 · HTML: https://mlsanity.com/paper/2202.07022/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Memory-Efficient Backpropagation Through Time](https://mlsanity.com/paper/1606.03401/) (2016)
- [Coarse-scale PDEs from fine-scale observations via machine learning](https://mlsanity.com/paper/1909.05707/) (2019)
- [Constructing coarse-scale bifurcation diagrams from spatio-temporal observations of microscopic simulations: A parsimonious machine learning approach](https://mlsanity.com/paper/2201.13323/) (2022)
- [Architectural Complexity Measures of Recurrent Neural Networks](https://mlsanity.com/paper/1602.08210/) (2016)
- [An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting](https://mlsanity.com/paper/1705.04378/) (2017)
- [A Nonlinear Dimensionality Reduction Framework Using Smooth Geodesics](https://mlsanity.com/paper/1707.06757/) (2017)
- [Bounded Manifold Completion](https://mlsanity.com/paper/1912.09026/) (2019)
- [Identifying manifolds underlying group motion in Vicsek agents](https://mlsanity.com/paper/1508.02809/) (2015)
- [Reconstruction of Fragmented Trajectories of Collective Motion using Hadamard Deep Autoencoders](https://mlsanity.com/paper/2110.10428/) (2021)
- [Network Topology Mapping from Partial Virtual Coordinates and Graph Geodesics](https://mlsanity.com/paper/1809.03319/) (2018)
- [Dimensionality Reduction of Collective Motion by Principal Manifolds](https://mlsanity.com/paper/1508.03332/) (2015)
- [Detecting phase transitions in collective behavior using manifold's curvature](https://mlsanity.com/paper/1509.07078/) (2015)
- [Image Denoising Using the Geodesics' Gramian of the Manifold Underlying Patch-Space](https://mlsanity.com/paper/2010.07769/) (2020)

## Built on by (most cited first, 10 of 10)

- [Predicting Ordinary Differential Equations with Transformers](https://mlsanity.com/paper/2307.12617/) (2023)
- [Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data](https://mlsanity.com/paper/2410.04814/) (2024)
- [Koopman-informed recurrent neural networks](https://mlsanity.com/paper/2410.23467/) (2024)
- [On the effectiveness of neural priors in modeling dynamical systems](https://mlsanity.com/paper/2303.05728/) (2023)
- [Physics-informed Convolutional Recurrent Surrogate Model for Reservoir Simulation with Well Controls](https://mlsanity.com/paper/2305.09056/) (2023)
- [Neural Entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws](https://mlsanity.com/paper/2507.01795/) (2025)
- [Modeling Nonlinear Oscillator Networks Using Physics-Informed Hybrid Reservoir Computing](https://mlsanity.com/paper/2411.05867/) (2024)
- [Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions](https://mlsanity.com/paper/2506.05678/) (2025)
- [Entropy stable conservative flux form neural networks](https://mlsanity.com/paper/2411.01746/) (2024)
- [Latent-Lagrangian Neural Networks for Reduced Order Modeling of Non-autonomous Nonlinear Dynamical Systems](https://mlsanity.com/paper/2610.04723/) (2026)

## Abstract (arXiv)

Classical methods of solving spatiotemporal dynamical systems include statistical approaches such as autoregressive integrated moving average, which assume linear and stationary relationships between systems' previous outputs. Development and implementation of linear methods are relatively simple, but they often do not capture non-linear relationships in the data. Thus, artificial neural networks (ANNs) are receiving attention from researchers in analyzing and forecasting dynamical systems. Recurrent neural networks (RNN), derived from feed-forward ANNs, use internal memory to process variable-length sequences of inputs. This allows RNNs to applicable for finding solutions for a vast variety of problems in spatiotemporal dynamical systems. Thus, in this paper, we utilize RNNs to treat some specific issues associated with dynamical systems. Specifically, we analyze the performance of RNNs applied to three tasks: reconstruction of correct Lorenz solutions for a system with a formulation error, reconstruction of corrupted collective motion trajectories, and forecasting of streamflow time series possessing spikes, representing three fields, namely, ordinary differential equations, collective motion, and hydrological modeling, respectively. We train and test RNNs uniquely in each task to demonstrate the broad applicability of RNNs in reconstruction and forecasting the dynamics of dynamical systems.