mlsanityData to 7 Oct 2026

NavBench: Probing Multimodal Large Language Models for Embodied Navigation

arXiv 2506.01031 ↗First posted 1 Jun 2025cs.CVcited by 24

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated strong generalization in vision-language tasks, yet their ability to understand and act within embodied environments remains underexplored. We present NavBench, a benchmark to evaluate the embodied navigation capabilities of MLLMs under zero-shot settings. NavBench consists of two components: (1) navigation comprehension, assessed through three cognitively grounded tasks including global instruction alignment, temporal progress estimation, and local observation-action reasoning, covering 3,200 question-answer pairs; and (2) step-by-step execution in 432 episodes across 72 indoor scenes, stratified by spatial, cognitive, and execution complexity. To support real-world deployment, we introduce a pipeline that converts MLLMs' outputs into robotic actions. We evaluate both proprietary and open-source models, finding that GPT-4o performs well across tasks, while lighter open-source models succeed in simpler cases. Results also show that models with higher comprehension scores tend to achieve better execution performance. Providing map-based context improves decision accuracy, especially in medium-difficulty scenarios. However, most models struggle with temporal understanding, particularly in estimating progress during navigation, which may pose a key challenge.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 68 references that are in the map. First posted on arXiv 1 Jun 2025.

Builds on

59 of 68 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Are Large Vision Language Models Good Game Players?4 Mar
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  3. 2024SpatialBot: Precise Spatial Understanding with Vision Language Models19 Jun
  4. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec

Other references · 55

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
  1. 2019Towards VQA Models That Can Readcited by 1,696
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  10. 2023A Survey on Multimodal Large Language Modelscited by 898
  11. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  12. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  13. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  14. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  15. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  16. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  17. 2019Vision-and-Dialog Navigationcited by 246
  18. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  19. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  20. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  21. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  22. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  23. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  24. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  25. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  26. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  27. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  28. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  29. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  30. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
  31. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  32. 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
  33. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  34. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  35. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 64
  36. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  37. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  38. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 20.

24 papers.

Sort:

By area:

  1. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  2. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  3. 2026pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoningcited by 12
  4. 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigationcited by 6
  5. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigationcited by 6
  6. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agentscited by 5
  7. 2026ProxyWar: Dynamic Assessment of LLM Code Generation in Game Arenascited by 3
  8. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
  9. 2026Lifelong Embodied Navigation Learningcited by 3
  10. 2026From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Groundingcited by 2
  11. 2026Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigationcited by 2
  12. 2026IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarkingcited by 1
12 more
  1. 2026UrbanGround: From Local Perception to Spatial Agency in a Real-Scale Citycited by 1
  2. 2025OceanGym: A Benchmark Environment for Underwater Embodied Agentscited by 0
  3. 2026MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirrorcited by 0
  4. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  5. 2026Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Modelscited by 0
  6. 2026NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulationcited by 0
  7. 2026Goal-oriented Navigation Instruction Generation with Tour Video Priorscited by 0
  8. 2026ChainSpace: A Chained-Reasoning Paradigm for Spatial Intelligencecited by 0
  9. 2026MAP: A Benchmark on Multimodal Accessibility Planning for Real World Placescited by 0
  10. 2026PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AIcited by 0
  11. 2026RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigationcited by 0
  12. 2026UniTrackPLA: Unified Panorama-Language-Action Model for Instruction-Guided Navigation and Dynamic Person Trackingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Models
  2. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks
  3. 2025Embodied Navigation Foundation Model
  4. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
  5. 2026Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation
  6. 2026InsightMap: Structured Spatial Modeling for Embodied Multimodal Reasoning
  7. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
  8. 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation