mlsanityData to 14 Oct 2026

Interaction Networks for Learning about Objects, Relations and Physics

arXiv 1612.00222 ↗First posted 1 Dec 2016cs.AI cs.LGcited by 509

Not on the map. Why?

Cited by 23 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reasoning about objects, relations, and physics is central to human intelligence, and a key goal of artificial intelligence. Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system. Our model takes graphs as input, performs object- and relation-centric reasoning in a way that is analogous to a simulation, and is implemented using deep neural networks. We evaluate its ability to reason about several challenging physical domains: n-body problems, rigid-body collision, and non-rigid dynamics. Our results show it can be trained to accurately simulate the physical trajectories of dozens of objects over thousands of time steps, estimate abstract quantities such as energy, and generalize automatically to systems with different numbers and configurations of objects and relations. Our interaction network implementation is the first general-purpose, learnable physics engine, and a powerful general framework for reasoning about object and relations in a wide variety of complex real-world domains.

Key facts. Cited by 509 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 120; 2022: 117; 2023: 94; 2024: 73; 2025: 43; 2026: 60.

509 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  3. 2021How Attentive are Graph Attention Networks?cited by 835
  4. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  5. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  6. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  7. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 303
  8. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 267
  9. 2021Conditional Object-Centric Learning from Videocited by 248
  10. 2023Neural Operators for Accelerating Scientific Simulations and Designcited by 221
  11. 2021Towards Causal Representation Learningcited by 209
  12. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
38 more
  1. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  3. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  4. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  5. 2021Graph Learning: A Surveycited by 141
  6. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  7. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  8. 2021A Unified Lottery Ticket Hypothesis for Graph Neural Networkscited by 118
  9. 2021Geometric Deep Learning on Molecular Representationscited by 113
  10. 2021PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physicscited by 103
  11. 2022Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvaturecited by 98
  12. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  13. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 92
  14. 2021Learned Coarse Models for Efficient Turbulence Simulationcited by 91
  15. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  16. 2023Everything is Connected: Graph Neural Networkscited by 85
  17. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 84
  18. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 82
  19. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  20. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  21. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  22. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  23. 2022Expander Graph Propagationcited by 75
  24. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  25. 2021Learning to Compose Visual Relationscited by 72
  26. 2022RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networkscited by 72
  27. 2022Rediscovering orbital mechanics with machine learningcited by 71
  28. 2021Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalizationcited by 70
  29. 2021Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modelingcited by 69
  30. 2022Equivariant Graph Mechanics Networks with Constraintscited by 68
  31. 2021Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systemscited by 67
  32. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 65
  33. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
  34. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  35. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  36. 2021Universal Approximation of Functions on Setscited by 62
  37. 2022Theory of Graph Neural Networks: Representation and Learningcited by 61
  38. 2022Anti-Symmetric DGN: a stable architecture for Deep Graph Networkscited by 60

50 most cited of 509 All 509 (JSON)