# Manipulation of Granular Materials by Learning Particle Interactions

arXiv: https://arxiv.org/abs/2111.02274 · HTML: https://mlsanity.com/paper/2111.02274/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Flexible Neural Representation for Physics Prediction](https://mlsanity.com/paper/1806.08047/) (2018)
- [Learning Visible Connectivity Dynamics for Cloth Smoothing](https://mlsanity.com/paper/2105.10389/) (2021)
- [Visual Closed-Loop Control for Pouring Liquids](https://mlsanity.com/paper/1610.02610/) (2016)
- [Inferring the Material Properties of Granular Media for Robotic Tasks](https://mlsanity.com/paper/2003.08032/) (2020)
- [Learning Robotic Manipulation of Granular Media](https://mlsanity.com/paper/1709.02833/) (2017)
- [Robot Gaining Accurate Pouring Skills through Self-Supervised Learning and Generalization](https://mlsanity.com/paper/2011.10150/) (2020)
- [Uncertainty-Aware Self-Supervised Target-Mass Grasping of Granular Foods](https://mlsanity.com/paper/2105.12946/) (2021)
- [Accurate Pouring with an Autonomous Robot Using an RGB-D Camera](https://mlsanity.com/paper/1810.03303/) (2018)
- [Ab Initio Particle-based Object Manipulation](https://mlsanity.com/paper/2107.08865/) (2021)
- [Model-free vision-based shaping of deformable plastic materials](https://mlsanity.com/paper/2001.11196/) (2020)
- [Learning to Manipulate Amorphous Materials](https://mlsanity.com/paper/2103.02533/) (2021)
- [Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates](https://mlsanity.com/paper/1611.03819/) (2016)
- [Motion Planning for Fluid Manipulation using Simplified Dynamics](https://mlsanity.com/paper/1603.02347/) (2016)

## Built on by (most cited first, 7 of 7)

- [Dynamic-Resolution Model Learning for Object Pile Manipulation](https://mlsanity.com/paper/2306.16700/) (2023)
- [Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization](https://mlsanity.com/paper/2207.14561/) (2022)
- [Tactile-based Object Retrieval From Granular Media](https://mlsanity.com/paper/2402.04536/) (2024)
- [GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks](https://mlsanity.com/paper/2503.05817/) (2025)
- [Localized Graph-Based Neural Dynamics Models for Terrain Manipulation](https://mlsanity.com/paper/2503.23270/) (2025)
- [Interactive Shaping of Granular Media Using Reinforcement Learning](https://mlsanity.com/paper/2509.06469/) (2025)
- [TRACE: A spatiotemporal contact memory graph network simulator for granular dynamics](https://mlsanity.com/paper/2609.02991/) (2026)

## Abstract (arXiv)

Manipulation of granular materials such as sand or rice remains an unsolved problem due to challenges such as the difficulty of defining their configuration or modeling the materials and their particles interactions. Current approaches tend to simplify the material dynamics and omit the interactions between the particles. In this paper, we propose to use a graph-based representation to model the interaction dynamics of the material and rigid bodies manipulating it. This allows the planning of manipulation trajectories to reach a desired configuration of the material. We use a graph neural network (GNN) to model the particle interactions via message-passing. To plan manipulation trajectories, we propose to minimise the Wasserstein distance between a predicted distribution of granular particles and their desired configuration. We demonstrate that the proposed method is able to pour granular materials into the desired configuration both in simulated and real scenarios.