# Combining Physics and Deep Learning to learn Continuous-Time Dynamics Models

arXiv: https://arxiv.org/abs/2110.01894 · HTML: https://mlsanity.com/paper/2110.01894/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 117 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 30 of 57)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Incorporating Symmetry into Deep Dynamics Models for Improved Generalization](https://mlsanity.com/paper/2002.03061/) (2020)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Hamiltonian Graph Networks with ODE Integrators](https://mlsanity.com/paper/1909.12790/) (2019)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)

## Built on by (most cited first, 20 of 20)

- [A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic](https://mlsanity.com/paper/2204.06362/) (2022)
- [Neural Autoencoder-Based Structure-Preserving Model Order Reduction and Control Design for High-Dimensional Physical Systems](https://mlsanity.com/paper/2312.06256/) (2023)
- [A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics](https://mlsanity.com/paper/2410.18868/) (2024)
- [Morphological-Symmetry-Equivariant Heterogeneous Graph Neural Network for Robotic Dynamics Learning](https://mlsanity.com/paper/2412.01297/) (2024)
- [A Black-Box Physics-Informed Estimator based on Gaussian Process Regression for Robot Inverse Dynamics Identification](https://mlsanity.com/paper/2310.06585/) (2023)
- [Generalizable and Fast Surrogates: Model Predictive Control of Articulated Soft Robots using Physics-Informed Neural Networks](https://mlsanity.com/paper/2502.01916/) (2025)
- [A Differential Dynamic Programming Framework for Inverse Reinforcement Learning](https://mlsanity.com/paper/2407.19902/) (2024)
- [Context-Aware Deep Lagrangian Networks for Model Predictive Control](https://mlsanity.com/paper/2506.15249/) (2025)
- [Bounding Distributional Shifts in World Modeling through Novelty Detection](https://mlsanity.com/paper/2508.06096/) (2025)
- [Learning Hamiltonian Dynamics at Scale: A Differential-Geometric Approach](https://mlsanity.com/paper/2509.24627/) (2025)
- [KFTD: Koopman-Fourier Time-Differentiable Network for Continuous Ocean Spatiotemporal Forecasting](https://mlsanity.com/paper/2606.17070/) (2026)
- [Newtonian and Lagrangian Neural Networks: A Comparison Towards Efficient Inverse Dynamics Identification](https://mlsanity.com/paper/2506.17994/) (2025)
- [Bayesian Inference and Learning in Nonlinear Dynamical Systems: A Framework for Incorporating Explicit and Implicit Prior Knowledge](https://mlsanity.com/paper/2508.15345/) (2025)
- [KeyCLD: Learning Constrained Lagrangian Dynamics in Keypoint Coordinates from Images](https://mlsanity.com/paper/2206.11030/) (2022)
- [Pseudo-rigid body networks: learning interpretable deformable object dynamics from partial observations](https://mlsanity.com/paper/2307.07975/) (2023)
- [Joint torques prediction of a robotic arm using neural networks](https://mlsanity.com/paper/2405.00695/) (2024)
- [ServoLNN: Lagrangian Neural Networks Driven by Servomechanisms](https://mlsanity.com/paper/2502.19802/) (2025)
- [Real-Time Structural Deflection Estimation in Hydraulically Actuated Systems Using 3D Flexible Multibody Simulation and DNNs](https://mlsanity.com/paper/2503.07528/) (2025)
- [Modelling of Underwater Vehicles using Physics-Informed Neural Networks with Control](https://mlsanity.com/paper/2504.20019/) (2025)
- [A Bayesian Filtering Approach for Learning Lagrangian Dynamics from Noisy Measurements](https://mlsanity.com/paper/2606.31137/) (2026)

## Abstract (arXiv)

Deep learning has been widely used within learning algorithms for robotics. One disadvantage of deep networks is that these networks are black-box representations. Therefore, the learned approximations ignore the existing knowledge of physics or robotics. Especially for learning dynamics models, these black-box models are not desirable as the underlying principles are well understood and the standard deep networks can learn dynamics that violate these principles. To learn dynamics models with deep networks that guarantee physically plausible dynamics, we introduce physics-inspired deep networks that combine first principles from physics with deep learning. We incorporate Lagrangian mechanics within the model learning such that all approximated models adhere to the laws of physics and conserve energy. Deep Lagrangian Networks (DeLaN) parametrize the system energy using two networks. The parameters are obtained by minimizing the squared residual of the Euler-Lagrange differential equation. Therefore, the resulting model does not require specific knowledge of the individual system, is interpretable, and can be used as a forward, inverse, and energy model. Previously these properties were only obtained when using system identification techniques that require knowledge of the kinematic structure. We apply DeLaN to learning dynamics models and apply these models to control simulated and physical rigid body systems. The results show that the proposed approach obtains dynamics models that can be applied to physical systems for real-time control. Compared to standard deep networks, the physics-inspired models learn better models and capture the underlying structure of the dynamics.