mlsanityData to 7 Oct 2026

LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestration

arXiv 2609.15098 ↗First posted 14 Sep 2026cs.CV cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Continuous-environment vision-and-language navigation (VLN-CE) requires interpreting natural-language instructions in unseen 3D environments and executing continuous low-level actions. Existing methods often depend on LiDAR, panoramic cameras, or extra sensors; separate geometric-mapping and semantic-navigation visual representations can cause long-trajectory spatial-semantic inconsistencies. We propose LG-VLN, a monocular zero-shot framework with shared visual features and LangGraph-based state orchestration. An online feed-forward 3D reconstruction network predicts depth, camera poses, and dense point clouds for agent-pose estimation and global map fusion. Geometry and navigation share dense CleanDIFT features: semantic consistency rejects incorrect inter-frame correspondences, while target-instance constraints define visual references whose similarity combines with local BLIP-2 image-text relevance to form a semantic value map. LangGraph represents instruction parsing, geometric perception, semantic value updates, path planning, action execution, and failure recovery as a directed state graph with conditional transitions, persistent state, and modular recovery mechanisms. On a fixed 550-episode subset of the R2R-CE val-unseen split, LG-VLN achieves 21.3% success and 12.1% success weighted by path length. Ablations show shared semantic features improve navigation, further boosted by combining visual similarity and image-text relevance. Results establish shared visual representations and explicit state orchestration as effective for zero-shot VLN-CE using monocular RGB alone. Code will be publicly released for reproducibility.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 56 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

47 of 56 references.

References · 47

  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  10. 2024Depth Anything V2cited by 1,385
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
35 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  3. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  4. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  5. 2023Emergent Correspondence from Image Diffusioncited by 437
  6. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  7. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  8. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  9. 2020Integrated Task and Motion Planningcited by 281
  10. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  11. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  12. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  13. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  14. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  15. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  16. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  17. 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 130
  18. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  19. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  20. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  21. 2016Revisiting Active Perceptioncited by 90
  22. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
  23. 2024Language Agents as Optimizable Graphscited by 81
  24. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  25. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  26. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
  27. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  28. 2020A Survey of Behavior Trees in Robotics and AIcited by 53
  29. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  30. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  31. 2024MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chainscited by 25
  32. 2025MonoDream: Monocular Vision-Language Navigation with Panoramic Dreamingcited by 15
  33. 2025UniForward: Unified 3D Scene and Semantic Field Reconstruction via Feed-Forward Gaussian Splatting from Only Sparse-View Imagescited by 13
  34. 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harnesscited by 6
  35. 2026MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLNcited by 1

Built on by

Not cited yet.