# Sequential Manipulation Planning on Scene Graph

arXiv: https://arxiv.org/abs/2207.04364 · HTML: https://mlsanity.com/paper/2207.04364/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 56 references that are in the map.
- First posted on arXiv 10 Jul 2022.

## Builds on (most cited first, 23 of 23)

- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences](https://mlsanity.com/paper/2103.14898/) (2021)
- [PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning](https://mlsanity.com/paper/1802.08705/) (2018)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds](https://mlsanity.com/paper/2104.03437/) (2021)
- [Human-centric Indoor Scene Synthesis Using Stochastic Grammar](https://mlsanity.com/paper/1808.08473/) (2018)
- [Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense](https://mlsanity.com/paper/1909.01507/) (2019)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1808.02201/) (2018)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities](https://mlsanity.com/paper/2007.15781/) (2020)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [FFRob: Leveraging Symbolic Planning for Efficient Task and Motion Planning](https://mlsanity.com/paper/1608.01335/) (2016)
- [Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars](https://mlsanity.com/paper/1704.00112/) (2017)
- [Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments](https://mlsanity.com/paper/2103.16095/) (2021)
- [Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective](https://mlsanity.com/paper/2108.01259/) (2021)
- [Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs](https://mlsanity.com/paper/2004.12248/) (2020)
- [Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations](https://mlsanity.com/paper/2108.01264/) (2021)
- [Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction](https://mlsanity.com/paper/1806.03497/) (2018)
- [Probabilistic Framework for Constrained Manipulations and Task and Motion Planning under Uncertainty](https://mlsanity.com/paper/2003.04259/) (2020)

## Built on by (most cited first, 17 of 17)

- [LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning](https://mlsanity.com/paper/2403.11552/) (2024)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [SGAligner : 3D Scene Alignment with Scene Graphs](https://mlsanity.com/paper/2304.14880/) (2023)
- ["Where am I?" Scene Retrieval with Language](https://mlsanity.com/paper/2404.14565/) (2024)
- [Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs](https://mlsanity.com/paper/2305.08522/) (2023)
- [VeriGraph: Scene Graphs for Execution Verifiable Robot Planning](https://mlsanity.com/paper/2411.10446/) (2024)
- [ESCA: Contextualizing Embodied Agents via Scene-Graph Generation](https://mlsanity.com/paper/2510.15963/) (2025)
- [Rearrange Indoor Scenes for Human-Robot Co-Activity](https://mlsanity.com/paper/2303.05676/) (2023)
- [BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation](https://mlsanity.com/paper/2406.10093/) (2024)
- [SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2412.11026/) (2024)
- [Vision Language Models Cannot Plan, but Can They Formalize?](https://mlsanity.com/paper/2509.21576/) (2025)
- [VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning](https://mlsanity.com/paper/2602.00637/) (2026)
- [GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning](https://mlsanity.com/paper/2609.19315/) (2026)
- [SAGE: Scene Graph-Aware Guidance and Execution for Long-Horizon Manipulation Tasks](https://mlsanity.com/paper/2509.21928/) (2025)
- [PARSE: Part-Aware Relational Spatial Modeling](https://mlsanity.com/paper/2603.07704/) (2026)
- [Grounding Vision and Language to 3D Masks for Long-Horizon Box Rearrangement](https://mlsanity.com/paper/2603.23676/) (2026)
- [Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework](https://mlsanity.com/paper/2610.17089/) (2026)

## Abstract (arXiv)

We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a stochastic optimization method. A task plan is then initialized by computing the Graph Editing Distance (GED) between the initial contact graphs and the goal configurations, which generates graph edit operations corresponding to possible robot actions. We finalize the task plan by imposing constraints to regulate the temporal feasibility of graph edit operations, ensuring valid task and motion correspondences. In a series of simulations and experiments, robots successfully complete complex sequential object rearrangement tasks that are difficult to specify using conventional planning language like Planning Domain Definition Language (PDDL), demonstrating the high feasibility and potential of robot sequential task planning on contact graph.