# Flexible Networks for Learning Physical Dynamics of Deformable Objects

arXiv: https://arxiv.org/abs/2112.03728 · HTML: https://mlsanity.com/paper/2112.03728/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Flexible Neural Representation for Physics Prediction](https://mlsanity.com/paper/1806.08047/) (2018)
- [Visual Grounding of Learned Physical Models](https://mlsanity.com/paper/2004.13664/) (2020)
- [PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing](https://mlsanity.com/paper/1910.08287/) (2019)
- [Factorised Neural Relational Inference for Multi-Interaction Systems](https://mlsanity.com/paper/1905.08721/) (2019)
- [Scalable Graph Networks for Particle Simulations](https://mlsanity.com/paper/2010.06948/) (2020)
- [Spatio-temporal Graph-RNN for Point Cloud Prediction](https://mlsanity.com/paper/2102.07482/) (2021)
- [Discovering Latent Representations of Relations for Interacting Systems](https://mlsanity.com/paper/2111.05514/) (2021)

## Built on by (most cited first, 1 of 1)

- [Grounding Graph Network Simulators using Physical Sensor Observations](https://mlsanity.com/paper/2302.11864/) (2023)

## Abstract (arXiv)

Learning the physical dynamics of deformable objects with particle-based representation has been the objective of many computational models in machine learning. While several state-of-the-art models have achieved this objective in simulated environments, most existing models impose a precondition, such that the input is a sequence of ordered point sets. That is, the order of the points in each point set must be the same across the entire input sequence. This precondition restrains the model from generalizing to real-world data, which is considered to be a sequence of unordered point sets. In this paper, we propose a model named time-wise PointNet (TP-Net) that solves this problem by directly consuming a sequence of unordered point sets to infer the future state of a deformable object with particle-based representation. Our model consists of a shared feature extractor that extracts global features from each input point set in parallel and a prediction network that aggregates and reasons on these features for future prediction. The key concept of our approach is that we use global features rather than local features to achieve invariance to input permutations and ensure the stability and scalability of our model. Experiments demonstrate that our model achieves state-of-the-art performance with real-time prediction speed in both synthetic dataset and real-world dataset. In addition, we provide quantitative and qualitative analysis on why our approach is more effective and efficient than existing approaches.