mlsanityData to 14 Oct 2026

RANa: Retrieval-Augmented Navigation

arXiv 2504.03524 ↗First posted 4 Apr 2025cs.CV cs.IR cs.ROcited by 2

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Methods for navigation based on large-scale learning typically treat each episode as a new problem, where the agent is spawned with a clean memory in an unknown environment. While these generalization capabilities to an unknown environment are extremely important, we claim that, in a realistic setting, an agent should have the capacity of exploiting information collected during earlier robot operations. We address this by introducing a new retrieval-augmented agent, trained with RL, capable of querying a database collected from previous episodes in the same environment and learning how to integrate this additional context information. We introduce a unique agent architecture for the general navigation task, evaluated on ImageNav, Instance-ImageNav and ObjectNav. Our retrieval and context encoding methods are data-driven and employ vision foundation models (FM) for both semantic and geometric understanding. We propose new benchmarks for these settings and we show that retrieval allows zero-shot transfer across tasks and environments while significantly improving performance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 75 of its 90 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

75 of 90 references.

References · 75

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  9. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  10. 2022A Generalist Agentcited by 808
  11. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  12. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
38 more
  1. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  2. 2018On Evaluation of Embodied Navigation Agentscited by 547
  3. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  4. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  6. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  7. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  8. 2020Learning to Explore using Active Neural SLAMcited by 316
  9. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
  10. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
  11. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  12. 2018Semi-parametric Topological Memory for Navigationcited by 209
  13. 2016Learning to Navigate in Complex Environmentscited by 198
  14. 2020Neural Topological SLAM for Visual Navigationcited by 195
  15. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  16. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  17. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  18. 2019Goal-conditioned Imitation Learningcited by 162
  19. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 160
  20. 2017Neural Episodic Controlcited by 151
  21. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  22. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
  23. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  24. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  25. 2019Scene Memory Transformer for Embodied Agents in Long-Horizon Taskscited by 107
  26. 2016Model-Free Episodic Controlcited by 102
  27. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  28. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  29. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 90
  30. 2024RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agentscited by 80
  31. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  32. 2022Offline Visual Representation Learning for Embodied Navigationcited by 75
  33. 2020The Marathon 2: A Navigation Systemcited by 70
  34. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70
  35. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 67
  36. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 67
  37. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 66
  38. 2024RTAB-Map as an Open-Source Lidar and Visual SLAM Library for Large-Scale and Long-Term Online Operationcited by 62

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
  2. 2026A Semantic Autonomy Framework for VLM-Integrated Indoor Mobile Robots: Hybrid Deterministic Reasoning and Cross-Robot Adaptive Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
  2. 2026Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation
  3. 2026Retrieval-Augmented Robots via Retrieve-Reason-Act
  4. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigation
  5. 2024REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environments
  6. 2026When Does Retrieval Help? A Study of In-Context Adaptation in Vision-Language-Action Models
  7. 2024STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning
  8. 2026STaR: Scalable Task-Conditioned Retrieval for Long-Horizon Multimodal Robot Memory