# EmbodiedRAG: Dynamic 3D Scene Graph Retrieval for Efficient and Scalable Robot Task Planning

arXiv: https://arxiv.org/abs/2410.23968 · HTML: https://mlsanity.com/paper/2410.23968/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2024.

## Built on by (most cited first, 24 of 24)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM](https://mlsanity.com/paper/2502.11142/) (2025)
- [MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning](https://mlsanity.com/paper/2505.03035/) (2025)
- [RoboAgent: Chaining Basic Capabilities for Embodied Task Planning](https://mlsanity.com/paper/2604.07774/) (2026)
- [FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning](https://mlsanity.com/paper/2604.03696/) (2026)
- [ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay](https://mlsanity.com/paper/2511.19033/) (2025)
- [Structured Interfaces for Automated Reasoning with 3D Scene Graphs](https://mlsanity.com/paper/2510.16643/) (2025)
- [LookPlanGraph: Embodied Instruction Following Method with VLM Graph Augmentation](https://mlsanity.com/paper/2512.21243/) (2025)
- [Can a Robot Walk the Robotic Dog: Triple-Zero Collaborative Navigation for Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2603.21723/) (2026)
- [LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigation](https://mlsanity.com/paper/2604.17190/) (2026)
- [R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space](https://mlsanity.com/paper/2512.15940/) (2025)
- [MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation](https://mlsanity.com/paper/2602.07082/) (2026)
- [Constrained Natural Language Action Planning for Resilient Embodied Systems](https://mlsanity.com/paper/2510.06357/) (2025)
- [RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion](https://mlsanity.com/paper/2512.18386/) (2025)
- [RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation](https://mlsanity.com/paper/2603.03745/) (2026)
- [TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation](https://mlsanity.com/paper/2605.01700/) (2026)
- [eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents](https://mlsanity.com/paper/2606.03374/) (2026)
- [ObsGraph: Hierarchical Observation Representation for Embodied Reasoning and Exploration](https://mlsanity.com/paper/2606.24068/) (2026)
- [DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors](https://mlsanity.com/paper/2607.00889/) (2026)
- [Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search](https://mlsanity.com/paper/2603.05642/) (2026)
- [TaskGround: Structured Executable Task Inference for Full-Scene Household Reasoning](https://mlsanity.com/paper/2605.18109/) (2026)
- [FRAME: Factored Retrieval via Attribute Readouts for Object-Centric Scene Memory](https://mlsanity.com/paper/2609.08886/) (2026)
- [Finder: Agentic Closed-Loop Object Finding for Embodied Grounding](https://mlsanity.com/paper/2609.18058/) (2026)
- [Token-Budgeted Candidate Retrieval for 3D Scene-Grounded VLMs: Category Names vs. Pairwise Geometry](https://mlsanity.com/paper/2610.14207/) (2026)

## Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have helped facilitate exciting progress for robotic planning in real, open-world environments. 3D scene graphs (3DSGs) offer a promising environment representation for grounding such LLM-based planners as they are compact and semantically rich. However, as the robot's environment scales (e.g., number of entities tracked) and the complexity of scene graph information increases (e.g., maintaining more attributes), providing the 3DSG as-is to an LLM-based planner quickly becomes infeasible due to input token count limits and attentional biases present in LLMs. Inspired by the successes of Retrieval-Augmented Generation (RAG) methods that retrieve query-relevant document chunks for LLM question and answering, we adapt the paradigm for our embodied domain. Specifically, we propose a 3D scene subgraph retrieval framework, called EmbodiedRAG, that we augment an LLM-based planner with for executing natural language robotic tasks. Notably, our retrieved subgraphs adapt to changes in the environment as well as changes in task-relevancy as the robot executes its plan. We demonstrate EmbodiedRAG's ability to significantly reduce input token counts (by an order of magnitude) and planning time (up to 70% reduction in average time per planning step) while improving success rates on AI2Thor simulated household tasks with a single-arm, mobile manipulator. Additionally, we implement EmbodiedRAG on a quadruped with a manipulator to highlight the performance benefits for robot deployment at the edge in real environments.