# Retrieval-Augmented Embodied Agents

arXiv: https://arxiv.org/abs/2404.11699 · HTML: https://mlsanity.com/paper/2404.11699/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2024.

## Built on by (most cited first, 30 of 31)

- [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024)
- [WorldEval: World Model as Real-World Robot Policies Evaluator](https://mlsanity.com/paper/2505.19017/) (2025)
- [ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge](https://mlsanity.com/paper/2505.21906/) (2025)
- [R+X: Retrieval and Execution from Everyday Human Videos](https://mlsanity.com/paper/2407.12957/) (2024)
- [Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation](https://mlsanity.com/paper/2409.14411/) (2024)
- [Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook](https://mlsanity.com/paper/2503.18016/) (2025)
- [Large Language Models for Multi-Robot Systems: A Survey](https://mlsanity.com/paper/2502.03814/) (2025)
- [ObjectVLA: End-to-End Open-World Object Manipulation Without Demonstration](https://mlsanity.com/paper/2502.19250/) (2025)
- [Discrete Policy: Learning Disentangled Action Space for Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2409.18707/) (2024)
- [REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environments](https://mlsanity.com/paper/2412.04759/) (2024)
- [MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?](https://mlsanity.com/paper/2406.19693/) (2024)
- [Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification](https://mlsanity.com/paper/2504.04578/) (2025)
- [ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval](https://mlsanity.com/paper/2511.06202/) (2025)
- [EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability](https://mlsanity.com/paper/2503.10042/) (2025)
- [From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model](https://mlsanity.com/paper/2605.22671/) (2026)
- [COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learning](https://mlsanity.com/paper/2508.01131/) (2025)
- [Retrieve-then-Steer: Online Success Memory for Test-Time Adaptation of Generative VLAs](https://mlsanity.com/paper/2605.10094/) (2026)
- [Chameleon: Control-Indexed Prospective Memory for Visuomotor Manipulation](https://mlsanity.com/paper/2603.24576/) (2026)
- [SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models](https://mlsanity.com/paper/2411.11323/) (2024)
- [Dejavu: Towards Experience Feedback Learning for Embodied Intelligence](https://mlsanity.com/paper/2510.10181/) (2025)
- [Retrieval-Augmented Robots via Retrieve-Reason-Act](https://mlsanity.com/paper/2603.02688/) (2026)
- [ECHO: Continuous Hierarchical Memory for Vision-Language-Action Models](https://mlsanity.com/paper/2605.10993/) (2026)
- [VLA-Pro: Cross-Task Procedural Memory Transfer for Vision-Language-Action Models](https://mlsanity.com/paper/2605.29562/) (2026)
- [RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models](https://mlsanity.com/paper/2409.12294/) (2024)
- [ActionSink: Toward Precise Robot Manipulation with Dynamic Integration of Action Flow](https://mlsanity.com/paper/2508.03218/) (2025)
- [Embodiment Transfer Learning for Vision-Language-Action Models](https://mlsanity.com/paper/2511.01224/) (2025)
- [Evolvable Embodied Agent for Robotic Manipulation via Long Short-Term Reflection and Optimization](https://mlsanity.com/paper/2604.13533/) (2026)
- [Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Making](https://mlsanity.com/paper/2604.07392/) (2026)
- [ASH: Agents that Self-Hone in Long-Horizon Worlds](https://mlsanity.com/paper/2605.14211/) (2026)
- [RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptation](https://mlsanity.com/paper/2608.25585/) (2026)

## Abstract (arXiv)

Embodied agents operating in complex and uncertain environments face considerable challenges. While some advanced agents handle complex manipulation tasks with proficiency, their success often hinges on extensive training data to develop their capabilities. In contrast, humans typically rely on recalling past experiences and analogous situations to solve new problems. Aiming to emulate this human approach in robotics, we introduce the Retrieval-Augmented Embodied Agent (RAEA). This innovative system equips robots with a form of shared memory, significantly enhancing their performance. Our approach integrates a policy retriever, allowing robots to access relevant strategies from an external policy memory bank based on multi-modal inputs. Additionally, a policy generator is employed to assimilate these strategies into the learning process, enabling robots to formulate effective responses to tasks. Extensive testing of RAEA in both simulated and real-world scenarios demonstrates its superior performance over traditional methods, representing a major leap forward in robotic technology.