mlsanityData to 14 Oct 2026

AnchorVLN: Geometry-Anchored Vision-Language Grounding Reasoning for Open-Vocabulary Navigation

arXiv 2609.12285 ↗First posted 10 Sep 2026cs.RO cs.CVcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Vision-Language Navigation (VLN) in unseen indoor environments is useful in real-world robotics, where an agent must follow natural-language instructions, locate objects, and answer spatial questions without a pre-built map or fixed object vocabulary. Multimodal vision-language models (VLMs) provide strong open-vocabulary grounding and zero-shot reasoning, but struggle to emit reliable metric quantities such as range, bearing, and comparative spatial relations directly from images. Existing approaches address this by folding geometry into hand-engineered pipelines or asking models to output waypoints, requiring changes to the control stack for different robots, tasks, or vocabularies. We introduce AnchorVLN, an open-vocabulary VLN system built on a simple rule: the VLM proposes semantics; geometry decides metrics. It is realised as EMBODIED-NAV-MCP, a Model Context Protocol (MCP) server driven by a VLM agent through a compact set of callable tools. Since no tool accepts distance in metres or bearing in radians, the schema enforces the semantic-geometry boundary without modifying the downstream autonomy stack. We benchmark both tasks of the CMU Vision-Language Navigation Challenge 2026: 30 instruction-following questions over 15 scenes and a frozen 45-question object-reference set. The full system achieves 64.4 percent on instruction following, dropping by 13.3 percentage points without controller modeling (t = 2.77). On object reference, geometric anchoring clears the challenge overlap threshold on 10 of 45 questions, versus 0 of 45 for direct coordinate estimation, reducing median center error from 3.37 m to 2.48 m.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 37 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

16 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models17 Jul
  2. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models26 May

Other references · 14

  1. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  2. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  3. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  4. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  5. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  6. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  7. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 65
  8. 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
  9. 2021Autonomous Exploration Development Environment and the Planning Algorithmscited by 27
  10. 2024VLA-3D: A Dataset for 3D Semantic Scene Understanding and Navigationcited by 21
  11. 2025IRef-VLA: A Benchmark for Interactive Referential Grounding with Imperfect Language in 3D Scenescited by 8
  12. 2026SysNav: Multi-Level Systematic Cooperation Enables Real-World, Cross-Embodiment Object Navigationcited by 4
2 more
  1. 2026Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLNcited by 3
  2. 2023AirLoc: Object-based Indoor Relocalizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OptiSight: Bridging Semantic Reasoning and Geometric Control for Embodied Navigation
  2. 2026Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation
  3. 2026Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation
  4. 2026OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms
  5. 2025FSR-VLN: Fast and Slow Reasoning for Vision-Language Navigation with Hierarchical Multi-modal Scene Graph
  6. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  7. 2026Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
  8. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agents