# Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks

arXiv: https://arxiv.org/abs/1807.09244 · HTML: https://mlsanity.com/paper/1807.09244/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2018.

## Built on by (most cited first, 13 of 13)

- [A Survey on Machine Learning Approaches for Modelling Intuitive Physics](https://mlsanity.com/paper/2202.06481/) (2022)
- [Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems](https://mlsanity.com/paper/2410.01376/) (2024)
- [Object and Relation Centric Representations for Push Effect Prediction](https://mlsanity.com/paper/2102.02100/) (2021)
- [AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge](https://mlsanity.com/paper/2504.01538/) (2025)
- [Breaking the Discretization Barrier of Continuous Physics Simulation Learning](https://mlsanity.com/paper/2509.17955/) (2025)
- [IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video](https://mlsanity.com/paper/2603.16432/) (2026)
- [Graph network for learning bi-directional physics](https://mlsanity.com/paper/2112.07054/) (2021)
- [Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos](https://mlsanity.com/paper/2106.04776/) (2021)
- [Fast Model-based Policy Search for Universal Policy Networks](https://mlsanity.com/paper/2202.05843/) (2022)
- [Toward Building Science Discovery Machines](https://mlsanity.com/paper/2103.15551/) (2021)
- [Intuitive Physics Guided Exploration for Sample Efficient Sim2real Transfer](https://mlsanity.com/paper/2104.08795/) (2021)
- [Can A Neural Network Hear the Shape of A Drum?](https://mlsanity.com/paper/2203.08073/) (2022)
- [Interpretable Representation Learning from Videos using Nonlinear Priors](https://mlsanity.com/paper/2410.18539/) (2024)

## Abstract (arXiv)

We propose a framework for the completely unsupervised learning of latent object properties from their interactions: the perception-prediction network (PPN). Consisting of a perception module that extracts representations of latent object properties and a prediction module that uses those extracted properties to simulate system dynamics, the PPN can be trained in an end-to-end fashion purely from samples of object dynamics. The representations of latent object properties learned by PPNs not only are sufficient to accurately simulate the dynamics of systems comprised of previously unseen objects, but also can be translated directly into human-interpretable properties (e.g., mass, coefficient of restitution) in an entirely unsupervised manner. Crucially, PPNs also generalize to novel scenarios: their gradient-based training can be applied to many dynamical systems and their graph-based structure functions over systems comprised of different numbers of objects. Our results demonstrate the efficacy of graph-based neural architectures in object-centric inference and prediction tasks, and our model has the potential to discover relevant object properties in systems that are not yet well understood.