mlsanityData to 7 Oct 2026

Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs

arXiv 2012.07277 ↗First posted 14 Dec 2020cs.ROcited by 83

Not on the map. Why?

Abstract (arXiv)

We present a visually grounded hierarchical planning algorithm for long-horizon manipulation tasks. Our algorithm offers a joint framework of neuro-symbolic task planning and low-level motion generation conditioned on the specified goal. At the core of our approach is a two-level scene graph representation, namely geometric scene graph and symbolic scene graph. This hierarchical representation serves as a structured, object-centric abstraction of manipulation scenes. Our model uses graph neural networks to process these scene graphs for predicting high-level task plans and low-level motions. We demonstrate that our method scales to long-horizon tasks and generalizes well to novel task goals. We validate our method in a kitchen storage task in both physical simulation and the real world. Our experiments show that our method achieved over 70% success rate and nearly 90% of subgoal completion rate on the real robot while being four orders of magnitude faster in computation time compared to standard search-based task-and-motion planner.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 16; 2023: 19; 2024: 15; 2025: 13; 2026: 11.

83 papers.

Sort:

By area:

  1. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  2. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 159
  3. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Modelscited by 62
  4. 2021Learning Symbolic Operators for Task and Motion Planningcited by 57
  5. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  6. 2022End-to-End Affordance Learning for Robotic Manipulationcited by 54
  7. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 51
  8. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
  9. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  10. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  11. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  12. 2022Continuous Scene Representations for Embodied AIcited by 39
38 more
  1. 2021Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordancescited by 38
  2. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 34
  3. 2023Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangementcited by 30
  4. 2025Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agentscited by 30
  5. 2023Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentationcited by 29
  6. 2022Reasoning with Scene Graphs for Robot Planning under Partial Observabilitycited by 25
  7. 20223D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphscited by 24
  8. 2023SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphscited by 24
  9. 2021Predicting Stable Configurations for Semantic Placement of Novel Objectscited by 23
  10. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  11. 2023Grounding Classical Task Planners via Vision-Language Modelscited by 23
  12. 2022Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planningcited by 19
  13. 2024A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approachescited by 19
  14. 2023Modeling Dynamic Environments with Scene Graph Memorycited by 18
  15. 2024Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Modelscited by 16
  16. 2023Primitive Skill-based Robot Learning from Human Evaluative Feedbackcited by 15
  17. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  18. 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
  19. 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planningcited by 14
  20. 2023Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphscited by 13
  21. 2025Compile Scene Graphs with Reinforcement Learningcited by 11
  22. 2021Learning to Search in Task and Motion Planning with Streamscited by 9
  23. 2022Learning Neuro-symbolic Programs for Language Guided Robot Manipulationcited by 9
  24. 2022Where To Start? Transferring Simple Skills to Complex Environmentscited by 9
  25. 2025MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoningcited by 9
  26. 2023NOIR: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 8
  27. 2023Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planningcited by 7
  28. 2023Translating Universal Scene Descriptions into Knowledge Graphs for Robotic Environmentcited by 7
  29. 2024VeriGraph: Scene Graphs for Execution Verifiable Robot Planningcited by 7
  30. 2021Learning to Design and Construct Bridge without Blueprintcited by 6
  31. 2021Discovering Generalizable Skills via Automated Generation of Diverse Taskscited by 5
  32. 2023Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Modelscited by 5
  33. 2024Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Priorcited by 5
  34. 2024BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulationcited by 5
  35. 2024Closed Loop Interactive Embodied Reasoning for Robot Manipulationcited by 4
  36. 2024Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modelingcited by 4
  37. 2022BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environmentcited by 3
  38. 2022Context-aware Mixture-of-Experts for Unbiased Scene Graph Generationcited by 3

50 most cited of 83 All 83 (JSON)