mlsanityData to 7 Oct 2026

GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation

arXiv 2404.06609 ↗First posted 9 Apr 2024cs.AI cs.ROcited by 93

Not on the map. Why?

Abstract (arXiv)

The Embodied AI community has made significant strides in visual navigation tasks, exploring targets from 3D coordinates, objects, language descriptions, and images. However, these navigation models often handle only a single input modality as the target. With the progress achieved so far, it is time to move towards universal navigation models capable of handling various goal types, enabling more effective user interaction with robots. To facilitate this goal, we propose GOAT-Bench, a benchmark for the universal navigation task referred to as GO to AnyThing (GOAT). In this task, the agent is directed to navigate to a sequence of targets specified by the category name, language description, or image in an open-vocabulary fashion. We benchmark monolithic RL and modular methods on the GOAT task, analyzing their performance across modalities, the role of explicit and implicit scene memories, their robustness to noise in goal specifications, and the impact of memory in lifelong scenarios.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 57 of its 62 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

57 of 62 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  10. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  11. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  12. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
38 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2018On Evaluation of Embodied Navigation Agentscited by 526
  3. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  4. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  5. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  7. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  8. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  9. 2017Embodied Question Answeringcited by 335
  10. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  11. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  12. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  13. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  14. 2020Neural Topological SLAM for Visual Navigationcited by 193
  15. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  16. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  17. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  18. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  19. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  20. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  21. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  22. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  23. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  24. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 104
  25. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  26. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
  27. 2023GOAT: GO to Any Thingcited by 89
  28. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  29. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
  30. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  31. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
  32. 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
  33. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53
  34. 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 52
  35. 2023Navigating to Objects Specified by Imagescited by 50
  36. 2021Learning to Map for Active Semantic Goal Navigationcited by 47
  37. 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 47
  38. 2020Semantic Curiosity for Active Visual Learningcited by 44

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 35; 2026: 44.

93 papers.

Sort:

By area:

  1. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
  2. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  3. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  4. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  5. 2024Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generationcited by 44
  6. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  7. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
  8. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  9. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  10. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  11. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
  12. 20253DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Modelcited by 25
38 more
  1. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  2. 2025Towards Physically Executable 3D Gaussian for Embodied Navigationcited by 16
  3. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  4. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  5. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  6. 2025Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answeringcited by 12
  7. 2024Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environmentscited by 11
  8. 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVscited by 11
  9. 2025FindingDory: A Benchmark to Evaluate Memory in Embodied Agentscited by 11
  10. 2024Task-oriented Sequential Grounding and Navigation in 3D Scenescited by 9
  11. 2024Multimodal LLM Guided Exploration and Active Mapping using Fisher Informationcited by 9
  12. 2025Safety of Embodied Navigation: A Surveycited by 9
  13. 2025LookOut: Real-World Humanoid Egocentric Navigationcited by 9
  14. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 7
  15. 2025TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goalscited by 7
  16. 2026Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agentscited by 7
  17. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigationcited by 6
  18. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigationcited by 6
  19. 2024Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insightscited by 5
  20. 2025Vision-Language Memory for Spatial Reasoningcited by 5
  21. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Explorationcited by 5
  22. 2025D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigationcited by 5
  23. 2026GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoningcited by 5
  24. 2024Zero-shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigationcited by 4
  25. 2025Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approachcited by 4
  26. 2025PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agentscited by 4
  27. 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
  28. 2025GeoExplorer: Active Geo-localization with Curiosity-Driven Explorationcited by 3
  29. 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigationcited by 3
  30. 2026Bridging the Indoor-Outdoor Gap: Vision-Centric Instruction-Guided Embodied Navigation for the Last Meterscited by 3
  31. 2026HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answeringcited by 3
  32. 2026FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answeringcited by 3
  33. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontierscited by 3
  34. 2026Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoningcited by 3
  35. 2026Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigationcited by 3
  36. 2025RANa: Retrieval-Augmented Navigationcited by 2
  37. 2025What does really matter in image goal navigation?cited by 2
  38. 2025Vision Language Models Cannot Plan, but Can They Formalize?cited by 2

50 most cited of 93 All 93 (JSON)