mlsanityData to 14 Oct 2026

Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments

arXiv 2412.10137 ↗First posted 13 Dec 2024cs.RO cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

We address the task of Vision-Language Navigation in Continuous Environments (VLN-CE) under the zero-shot setting. Zero-shot VLN-CE is particularly challenging due to the absence of expert demonstrations for training and minimal environment structural prior to guide navigation. To confront these challenges, we propose a Constraint-Aware Navigator (CA-Nav), which reframes zero-shot VLN-CE as a sequential, constraint-aware sub-instruction completion process. CA-Nav continuously translates sub-instructions into navigation plans using two core modules: the Constraint-Aware Sub-instruction Manager (CSM) and the Constraint-Aware Value Mapper (CVM). CSM defines the completion criteria for decomposed sub-instructions as constraints and tracks navigation progress by switching sub-instructions in a constraint-aware manner. CVM, guided by CSM's constraints, generates a value map on the fly and refines it using superpixel clustering to improve navigation stability. CA-Nav achieves the state-of-the-art performance on two VLN-CE benchmarks, surpassing the previous best method by 12 percent and 13 percent in Success Rate on the validation unseen splits of R2R-CE and RxR-CE, respectively. Moreover, CA-Nav demonstrates its effectiveness in real-world robot deployments across various indoor scenes and instructions.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 75 of its 87 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

75 of 87 references.

References · 75

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  10. 2024Depth Anything V2cited by 1,422
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
38 more
  1. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  2. 2023Large Language Modelscited by 780
  3. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  4. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  6. 2018On Evaluation of Embodied Navigation Agentscited by 547
  7. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  8. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  10. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  11. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  12. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  13. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  14. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  15. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  16. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  17. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  18. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  19. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  20. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  21. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  22. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  23. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  24. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  25. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  26. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  27. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  28. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  29. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  30. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  31. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  32. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  33. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
  34. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  35. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
  36. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  37. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  38. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 89

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 35.

50 papers.

Sort:

By area:

  1. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
  2. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  3. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 17
  4. 2025DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigationcited by 17
  5. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  6. 2026SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigationcited by 9
  7. 2025VLN-Zero: Rapid Exploration and Cache-Enabled Neurosymbolic Vision-Language Planning for Zero-Shot Transfer in Robot Navigationcited by 8
  8. 2026HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory Systemcited by 8
  9. 2026A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Modelcited by 8
  10. 2025Structured Preference Optimization for Vision-Language Long-Horizon Task Planningcited by 7
  11. 2025Active Test-time Vision-Language Navigationcited by 7
  12. 2025STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimizationcited by 6
38 more
  1. 2026Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigationcited by 6
  2. 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harnesscited by 6
  3. 2025D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigationcited by 5
  4. 2026Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Explorationcited by 5
  5. 2026One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representationcited by 5
  6. 2025Boosting Zero-Shot VLN via Abstract Obstacle Map-Based Waypoint Prediction with TopoGraph-and-VisitInfo-Aware Promptingcited by 4
  7. 2026Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigationcited by 4
  8. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  9. 2026Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigationcited by 3
  10. 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoningcited by 3
  11. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harnesscited by 3
  12. 2025Image Recognition with Online Lightweight Vision Transformer: A Surveycited by 2
  13. 2026Semantic Audio-Visual Navigation in Continuous Environmentscited by 2
  14. 2026SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigationcited by 2
  15. 2026P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigationcited by 2
  16. 2026GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigationcited by 2
  17. 2026Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigationcited by 2
  18. 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
  19. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigationcited by 2
  20. 2025Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigationcited by 1
  21. 2025User-Feedback-Driven Adaptation for Vision-and-Language Navigationcited by 1
  22. 2026LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMscited by 1
  23. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  24. 2026Navi-Agent: Unlocalized Monocular Navigation Agentcited by 1
  25. 2025TP-MDDN: Task-Preferenced Multi-Demand-Driven Navigation with Autonomous Decision-Makingcited by 0
  26. 2026SuReNav: Superpixel Graph-based Constraint Relaxation for Navigation in Over-constrained Environmentscited by 0
  27. 2026Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigationcited by 0
  28. 2026Rule-VLN: Bridging Perception and Compliance via Semantic Reasoning and Geometric Rectificationcited by 0
  29. 2026Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigationcited by 0
  30. 2026From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigationcited by 0
  31. 2026If, Then, Otherwise: Diagnosing Conditional Branching in Vision-Language Navigationcited by 0
  32. 2026One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypointscited by 0
  33. 2026LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestrationcited by 0
  34. 2026EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Modelscited by 0
  35. 2026RiverVLN: Phase-Grounded Temporal Vision--Language Navigation for Unmanned Surface Vehiclescited by 0
  36. 2026Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigationcited by 0
  37. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Methodcited by 0
  38. 2026Robust Vision-Language Navigation with Awareness of Instruction-Execution Progresscited by 0