mlsanityData to 14 Oct 2026

Mobile Robot Navigation Using Hand-Drawn Maps: A Vision Language Model Approach

arXiv 2502.00114 ↗First posted 31 Jan 2025cs.RO cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Hand-drawn maps can be used to convey navigation instructions between humans and robots in a natural and efficient manner. However, these maps can often contain inaccuracies such as scale distortions and missing landmarks which present challenges for mobile robot navigation. This paper introduces a novel Hand-drawn Map Navigation (HAM-Nav) architecture that leverages pre-trained vision language models (VLMs) for robot navigation across diverse environments, hand-drawing styles, and robot embodiments, even in the presence of map inaccuracies. HAM-Nav integrates a unique Selective Visual Association Prompting approach for topological map-based position estimation and navigation planning as well as a Predictive Navigation Plan Parser to infer missing landmarks. Extensive experiments were conducted in photorealistic simulated environments, using both wheeled and legged robots, demonstrating the effectiveness of HAM-Nav in terms of navigation success rates and Success weighted by Path Length. Furthermore, a user study in real-world environments highlighted the practical utility of hand-drawn maps for robot navigation as well as successful navigation outcomes compared against a non-hand-drawn map approach.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 52 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

19 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MLLM-Search: A Zero-Shot Approach to Finding People using Multimodal Large Language Models27 Nov

Other references · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  4. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  5. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  6. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  7. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 89
  8. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 66
  9. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 62
  10. 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
  11. 2024NavFormer: A Transformer Architecture for Robot Target-Driven Navigation in Unknown and Dynamic Environmentscited by 20
  12. 2021Deep Reinforcement Learning for Decentralized Multi-Robot Exploration With Macro Actionscited by 13
6 more
  1. 2024Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systemscited by 12
  2. 2020High-Speed Robot Navigation using Predicted Occupancy Mapscited by 10
  3. 2020Robot Navigation with Map-Based Deep Reinforcement Learningcited by 9
  4. 2024TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigationcited by 7
  5. 20244CNet: A Diffusion Approach to Map Prediction for Decentralized Multi-Robot Explorationcited by 3
  6. 2015Monte Carlo Localization in Hand-Drawn Mapscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  2. 2026SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planningcited by 3
  3. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2
  4. 2025SketchPlan: Diffusion Based Drone Planning From Human Sketchescited by 1
  5. 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026NaViRrator: Robot Navigation from Human-Readable Maps through a Learned Visual Route
  2. 2023Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models
  3. 2024CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments
  4. 2026Sim-to-Real Transfer of Vision-Language Navigation in Continuous Environments Using an Ackermann-Steered Mobile Robot
  5. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities
  6. 2026GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
  7. 2024Navigation with VLM framework: Towards Going to Any Language
  8. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Models