mlsanityData to 7 Oct 2026

Efficient and Interpretable Robot Manipulation with Graph Neural Networks

arXiv 2102.13177 ↗First posted 25 Feb 2021cs.RO cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Manipulation tasks, like loading a dishwasher, can be seen as a sequence of spatial constraints and relationships between different objects. We aim to discover these rules from demonstrations by posing manipulation as a classification problem over a graph, whose nodes represent task-relevant entities like objects and goals, and present a graph neural network (GNN) policy architecture for solving this problem from demonstrations. In our experiments, a single GNN policy trained using imitation learning (IL) on 20 expert demos can solve blockstacking, rearrangement, and dishwasher loading tasks; once the policy has learned the spatial structure, it can generalize to a larger number of objects, goal configurations, and from simulation to the real world. These experiments show that graphical IL can solve complex long-horizon manipulation problems without requiring detailed task descriptions. Videos can be found at: https://youtu.be/POxaTDAj7aY.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

26 of 45 references.

References · 26

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  6. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  7. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  8. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  9. 2015Gated Graph Sequence Neural Networkscited by 792
  10. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 735
  11. 2019Solving Rubik's Cube with a Robot Handcited by 699
  12. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
14 more
  1. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  2. 2020Integrated Task and Motion Planningcited by 281
  3. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  4. 2019Variational Graph Recurrent Neural Networkscited by 86
  5. 2018Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstrationcited by 60
  6. 2015Learning Visual Predictive Models of Physics for Playing Billiardscited by 58
  7. 2019Towards Practical Multi-Object Manipulation using Relational Reinforcement Learningcited by 54
  8. 2020Learning compositional models of robot skills for task and motion planningcited by 47
  9. 2020Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Imagecited by 45
  10. 2019Graph-Structured Visual Imitationcited by 43
  11. 2020Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networkscited by 38
  12. 2018Active model learning and diverse action sampling for task and motion planningcited by 28
  13. 2018Learning to guide task and motion planning using score-space representationcited by 27
  14. 2019Object-centric Forward Modeling for Model Predictive Controlcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 6; 2024: 1; 2025: 2; 2026: 4.

21 papers.

Sort:

By area:

  1. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 105
  2. 2022Graph Neural Networks in IoT: A Surveycited by 27
  3. 2022Policy Architectures for Compositional Generalization in Controlcited by 23
  4. 2023Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Modelscited by 16
  5. 2022Transformers are Adaptable Task Plannerscited by 15
  6. 2022Where To Start? Transferring Simple Skills to Complex Environmentscited by 9
  7. 2022Graph-based Reinforcement Learning meets Mixed Integer Programs: An application to 3D robot assembly discoverycited by 5
  8. 2023Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Modelscited by 5
  9. 2023Graph learning in robotics: a surveycited by 5
  10. 2023SceneScore: Learning a Cost Function for Object Arrangementcited by 4
  11. 2021What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networkscited by 2
  12. 2023Efficient and Feasible Robotic Assembly Sequence Planning via Graph Representation Learningcited by 2
9 more
  1. 2023Multi-Object Graph Affordance Network: Goal-Oriented Planning through Learned Compound Object Affordancescited by 2
  2. 2022Assembly Planning from Observations under Physical Constraintscited by 1
  3. 2025State Combinatorial Generalization In Decision Making With Conditional Diffusion Modelscited by 1
  4. 2024Interpretable Responsibility Sharing as a Heuristic for Task and Motion Planningcited by 0
  5. 2025ARM-Explainer -- Explaining and improving graph neural network predictions for the maximum clique problem using node features and association rule miningcited by 0
  6. 2026SWING: Unlocking Implicit Graph Representations for Graph Random Featurescited by 0
  7. 2026Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planningcited by 0
  8. 2026EUPHORIA: Efficient Universal Planning via Hybrid Optimization for Robust Industrial Robotic Assemblycited by 0
  9. 2026SIR: Structured Image Representations for Explainable Robot Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Offline Imitation Learning Through Graph Search and Retrieval
  2. 2021Geometric Task Networks: Learning Efficient and Explainable Skill Coordination for Object Manipulation
  3. 2025Compose by Focus: Scene Graph-based Atomic Skills
  4. 2025Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions
  5. 2026SparTa: Sparse Graphical Task Models from a Handful of Demonstrations
  6. 2021A Variational Graph Autoencoder for Manipulation Action Recognition and Prediction
  7. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
  8. 2022Factored World Models for Zero-Shot Generalization in Robotic Manipulation