mlsanityData to 7 Oct 2026

Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

arXiv 1810.01566 ↗First posted 3 Oct 2018cs.LG cs.AI cs.ROcited by 219

Not on the map. Why?

Abstract (arXiv)

Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.

Key facts. Cited by 219 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 32; 2023: 47; 2024: 41; 2025: 34; 2026: 37.

219 papers.

Sort:

By area:

  1. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  3. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  4. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  5. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  6. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 145
  7. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  8. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  9. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  10. 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
  11. 2021PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physicscited by 100
  12. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 92
38 more
  1. 2021Learned Coarse Models for Efficient Turbulence Simulationcited by 90
  2. 2022Predicting Physics in Mesh-reduced Space with Temporal Attentioncited by 82
  3. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 78
  4. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  5. 2022RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networkscited by 69
  6. 2022Equivariant Graph Mechanics Networks with Constraintscited by 67
  7. 2021DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contactcited by 64
  8. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  9. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  10. 2021DiffPD: Differentiable Projective Dynamicscited by 62
  11. 2021Learning Visible Connectivity Dynamics for Cloth Smoothingcited by 61
  12. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 61
  13. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  14. 2022Geometrically Equivariant Graph Neural Networks: A Surveycited by 55
  15. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  16. 2023Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformerscited by 54
  17. 2021An End-to-End Differentiable Framework for Contact-Aware Robot Designcited by 53
  18. 2023Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Propertiescited by 53
  19. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 52
  20. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  21. 2022DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Toolscited by 44
  22. 2022Learning rigid dynamics with face interaction graph networkscited by 43
  23. 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 42
  24. 2021Simulating Continuum Mechanics with Multi-Scale Graph Neural Networkscited by 39
  25. 2021Spline-PINN: Approaching PDEs without Data using Fast, Physics-Informed Hermite-Spline CNNscited by 38
  26. 2022Learning Physical Dynamics with Subequivariant Graph Neural Networkscited by 38
  27. 2022Physical Design using Differentiable Learned Simulatorscited by 37
  28. 2023FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulationcited by 37
  29. 2022NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fieldscited by 36
  30. 2022State-specific protein-ligand complex structure prediction with a multi-scale deep generative modelcited by 35
  31. 2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
  32. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  33. 2022Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamicscited by 30
  34. 2024Neural Material Adaptor for Visual Grounding of Intrinsic Dynamicscited by 30
  35. 2021Constructing Neural Network-Based Models for Simulating Dynamical Systemscited by 29
  36. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  37. 2021Constraint-based graph network simulatorcited by 27
  38. 2023LagrangeBench: A Lagrangian Fluid Mechanics Benchmarking Suitecited by 26

50 most cited of 219 All 219 (JSON)