# Distilling Governing Laws and Source Input for Dynamical Systems from Videos

arXiv: https://arxiv.org/abs/2205.01314 · HTML: https://mlsanity.com/paper/2205.01314/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 3 May 2022.

## Builds on (most cited first, 14 of 14)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects](https://mlsanity.com/paper/1806.01794/) (2018)
- [Learning to Decompose and Disentangle Representations for Video Prediction](https://mlsanity.com/paper/1806.04166/) (2018)
- [Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning](https://mlsanity.com/paper/2103.16564/) (2021)
- [Learning Visual Predictive Models of Physics for Playing Billiards](https://mlsanity.com/paper/1511.07404/) (2015)
- [Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video](https://mlsanity.com/paper/1905.11169/) (2019)
- [Structured Object-Aware Physics Prediction for Video Modeling and Planning](https://mlsanity.com/paper/1910.02425/) (2019)
- [Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images](https://mlsanity.com/paper/1511.04048/) (2015)
- [Symbolic Pregression: Discovering Physical Laws from Distorted Video](https://mlsanity.com/paper/2005.11212/) (2020)
- [Interpretable Intuitive Physics Model](https://mlsanity.com/paper/1808.10002/) (2018)
- [PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations](https://mlsanity.com/paper/1705.09805/) (2017)
- [Unsupervised Intuitive Physics from Visual Observations](https://mlsanity.com/paper/1805.05086/) (2018)
- [Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces](https://mlsanity.com/paper/1904.06827/) (2019)
- [Learning A Physical Long-term Predictor](https://mlsanity.com/paper/1703.00247/) (2017)

## Built on by (most cited first, 2 of 2)

- [Discovering Latent Response Laws in Forced Physical Systems](https://mlsanity.com/paper/2607.09801/) (2026)
- [Vision-based Discovery of Nonlinear Dynamics for 3D Moving Target](https://mlsanity.com/paper/2404.17865/) (2024)

## Abstract (arXiv)

Distilling interpretable physical laws from videos has led to expanded interest in the computer vision community recently thanks to the advances in deep learning, but still remains a great challenge. This paper introduces an end-to-end unsupervised deep learning framework to uncover the explicit governing equations of dynamics presented by moving object(s), based on recorded videos. Instead in the pixel (spatial) coordinate system of image space, the physical law is modeled in a regressed underlying physical coordinate system where the physical states follow potential explicit governing equations. A numerical integrator-based sparse regression module is designed and serves as a physical constraint to the autoencoder and coordinate system regression, and, in the meanwhile, uncover the parsimonious closed-form governing equations from the learned physical states. Experiments on simulated dynamical scenes show that the proposed method is able to distill closed-form governing equations and simultaneously identify unknown excitation input for several dynamical systems recorded by videos, which fills in the gap in literature where no existing methods are available and applicable for solving this type of problem.