mlsanityData to 7 Oct 2026

L3MVN: Leveraging Large Language Models for Visual Target Navigation

arXiv 2304.05501 ↗First posted 11 Apr 2023cs.ROcited by 118

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual target navigation in unknown environments is a crucial problem in robotics. Despite extensive investigation of classical and learning-based approaches in the past, robots lack common-sense knowledge about household objects and layouts. Prior state-of-the-art approaches to this task rely on learning the priors during the training and typically require significant expensive resources and time for learning. To address this, we propose a new framework for visual target navigation that leverages Large Language Models (LLM) to impart common sense for object searching. Specifically, we introduce two paradigms: (i) zero-shot and (ii) feed-forward approaches that use language to find the relevant frontier from the semantic map as a long-term goal and explore the environment efficiently. Our analysis demonstrates the notable zero-shot generalization and transfer capabilities from the use of language. Experiments on Gibson and Habitat-Matterport 3D (HM3D) demonstrate that the proposed framework significantly outperforms existing map-based methods in terms of success rate and generalization. Ablation analysis also indicates that the common-sense knowledge from the language model leads to more efficient semantic exploration. Finally, we provide a real robot experiment to verify the applicability of our framework in real-world scenarios. The supplementary video and code can be accessed via the following link: https://sites.google.com/view/l3mvn.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Apr 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 32; 2025: 38; 2026: 42.

118 papers.

Sort:

By area:

  1. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  2. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  3. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  4. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
  5. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  6. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  7. 2024RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesiscited by 55
  8. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  9. 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Modelscited by 46
  10. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
  11. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
  12. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
38 more
  1. 2024ReplanVLM: Replanning Robotic Tasks with Visual Language Modelscited by 28
  2. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
  3. 2023ThinkBot: Embodied Instruction Following with Thought Chain Reasoningcited by 19
  4. 2024Zero-shot Object Navigation with Vision-Language Models Reasoningcited by 18
  5. 2024Verifiably Following Complex Robot Instructions with Foundation Modelscited by 15
  6. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  7. 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
  8. 2024Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robotscited by 14
  9. 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 13
  10. 2025Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directionscited by 12
  11. 2025NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistantscited by 11
  12. 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVscited by 11
  13. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  14. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  15. 2023Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Modelscited by 8
  16. 2023Multimodal Large Language Model for Visual Navigationcited by 8
  17. 2024Embodied Instruction Following in Unknown Environmentscited by 8
  18. 2025PanoNav: Mapless Zero-Shot Object Navigation with Panoramic Scene Parsing and Dynamic Memorycited by 8
  19. 2024How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments?cited by 7
  20. 2024CAMON: Cooperative Agents for Multi-Object Navigation with LLM-based Conversationscited by 7
  21. 2024Generative AI Agents in Autonomous Machines: A Safety Perspectivecited by 7
  22. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 7
  23. 2025FlySearch: Exploring how vision-language models explorecited by 7
  24. 2024Can LLMs Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesiscited by 6
  25. 2024MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigationcited by 6
  26. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigationcited by 6
  27. 2025Learning to Tune Like an Expert: Interpretable and Scene-Aware Navigation via MLLM Reasoning and CVAE-Based Adaptationcited by 6
  28. 2025VISTAv2: World Imagination for Indoor Vision-and-Language Navigationcited by 6
  29. 2024DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenescited by 5
  30. 2025ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigationcited by 5
  31. 2025SayCoNav: Utilizing Large Language Models for Adaptive Collaboration in Decentralized Multi-Robot Navigationcited by 5
  32. 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrievalcited by 5
  33. 2025IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigationcited by 5
  34. 2026APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigationcited by 5
  35. 2026LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigationcited by 5
  36. 2024DOZE: A Dataset for Open-Vocabulary Zero-Shot Object Navigation in Dynamic Environmentscited by 4
  37. 2024Find Everything: A General Vision Language Model Approach to Multi-Object Searchcited by 4
  38. 2024MLLM-Search: A Zero-Shot Approach to Finding People using Multimodal Large Language Modelscited by 4

50 most cited of 118 All 118 (JSON)