# Constructing Neural Network-Based Models for Simulating Dynamical Systems

arXiv: https://arxiv.org/abs/2111.01495 · HTML: https://mlsanity.com/paper/2111.01495/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 78 of its 141 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 30 of 78)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Deep Learning on Graphs: A Survey](https://mlsanity.com/paper/1812.04202/) (2018)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)

## Built on by (most cited first, 30 of 31)

- [RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks](https://mlsanity.com/paper/2205.02909/) (2022)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing](https://mlsanity.com/paper/2407.01418/) (2024)
- [Stable Port-Hamiltonian Neural Networks](https://mlsanity.com/paper/2502.02480/) (2025)
- [Meta-Learning of Neural State-Space Models Using Data From Similar Systems](https://mlsanity.com/paper/2211.07768/) (2022)
- [PINNSim: A Simulator for Power System Dynamics based on Physics-Informed Neural Networks](https://mlsanity.com/paper/2303.10256/) (2023)
- [From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems](https://mlsanity.com/paper/2405.11542/) (2024)
- [Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs](https://mlsanity.com/paper/2205.12550/) (2022)
- [Learning Neural Differential Algebraic Equations via Operator Splitting](https://mlsanity.com/paper/2403.12938/) (2024)
- [Meta-Learning for Physically-Constrained Neural System Identification](https://mlsanity.com/paper/2501.06167/) (2025)
- [AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecture](https://mlsanity.com/paper/2506.06580/) (2025)
- [MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models](https://mlsanity.com/paper/2404.12097/) (2024)
- [Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations](https://mlsanity.com/paper/2410.10174/) (2024)
- [Combining Federated Learning and Control: A Survey](https://mlsanity.com/paper/2407.11069/) (2024)
- [Hard-Constrained Neural Networks with Physics-Embedded Architecture for Residual Dynamics Learning and Invariant Enforcement in Cyber-Physical Systems](https://mlsanity.com/paper/2511.23307/) (2025)
- [Surface Similarity Parameter: A New Machine Learning Loss Metric for Oscillatory Spatio-Temporal Data](https://mlsanity.com/paper/2204.06843/) (2022)
- [Message Propagation Through Time: An Algorithm for Sequence Dependency Retention in Time Series Modeling](https://mlsanity.com/paper/2309.16882/) (2023)
- [Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines](https://mlsanity.com/paper/2406.07572/) (2024)
- [Double projection for reconstructing dynamical systems: between stochastic and deterministic regimes](https://mlsanity.com/paper/2510.01089/) (2025)
- [A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series](https://mlsanity.com/paper/2511.06609/) (2025)
- [Identifying Simulation Model Through Alternative Techniques for a Medical Device Assembly Process](https://mlsanity.com/paper/2309.15094/) (2023)
- [Machine-Learned Closure of URANS for Stably Stratified Turbulence: Connecting Physical Timescales & Data Hyperparameters of Deep Time-Series Models](https://mlsanity.com/paper/2404.16141/) (2024)
- [Fast Deep Predictive Coding Networks for Videos Feature Extraction without Labels](https://mlsanity.com/paper/2409.04945/) (2024)
- [ServoLNN: Lagrangian Neural Networks Driven by Servomechanisms](https://mlsanity.com/paper/2502.19802/) (2025)
- [Physics-informed Modularized Neural Network for Advanced Building Control by Deep Reinforcement Learning](https://mlsanity.com/paper/2504.05397/) (2025)
- [The Dissipation Theory of Aging: A Quantitative Analysis Using a Cellular Aging Map](https://mlsanity.com/paper/2504.13044/) (2025)
- [Comparison of neural network training strategies for the simulation of dynamical systems](https://mlsanity.com/paper/2512.03851/) (2025)
- [Dual-Network PINNs for Optimal Control: A Reproducible Benchmark on the Mass-Spring-Damper System](https://mlsanity.com/paper/2606.15271/) (2026)
- [Safe-by-Design Learning via Energy-based Neural Networks](https://mlsanity.com/paper/2609.36942/) (2026)
- [Variational Augmented Invertible Koopman Autoencoder for probabilistic time series forecasting](https://mlsanity.com/paper/2609.37435/) (2026)

## Abstract (arXiv)

Dynamical systems see widespread use in natural sciences like physics, biology, chemistry, as well as engineering disciplines such as circuit analysis, computational fluid dynamics, and control. For simple systems, the differential equations governing the dynamics can be derived by applying fundamental physical laws. However, for more complex systems, this approach becomes exceedingly difficult. Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. In recent years, there has been an increased interest in data-driven modeling techniques, in particular neural networks have proven to provide an effective framework for solving a wide range of tasks. This paper provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome. Based on the reviewed literature and identified challenges, we provide a discussion on promising research areas.