LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestration
Not on the map. Why?
Abstract (arXiv)
Continuous-environment vision-and-language navigation (VLN-CE) requires interpreting natural-language instructions in unseen 3D environments and executing continuous low-level actions. Existing methods often depend on LiDAR, panoramic cameras, or extra sensors; separate geometric-mapping and semantic-navigation visual representations can cause long-trajectory spatial-semantic inconsistencies. We propose LG-VLN, a monocular zero-shot framework with shared visual features and LangGraph-based state orchestration. An online feed-forward 3D reconstruction network predicts depth, camera poses, and dense point clouds for agent-pose estimation and global map fusion. Geometry and navigation share dense CleanDIFT features: semantic consistency rejects incorrect inter-frame correspondences, while target-instance constraints define visual references whose similarity combines with local BLIP-2 image-text relevance to form a semantic value map. LangGraph represents instruction parsing, geometric perception, semantic value updates, path planning, action execution, and failure recovery as a directed state graph with conditional transitions, persistent state, and modular recovery mechanisms. On a fixed 550-episode subset of the R2R-CE val-unseen split, LG-VLN achieves 21.3% success and 12.1% success weighted by path length. Ablations show shared semantic features improve navigation, further boosted by combining visual similarity and image-text relevance. Results establish shared visual representations and explicit state orchestration as effective for zero-shot VLN-CE using monocular RGB alone. Code will be publicly released for reproducibility.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 56 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
47 of 56 references.
References · 47
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2024Depth Anything V2cited by 1,385
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
35 more
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2025Continuous 3D Perception Model with Persistent Statecited by 527
- 2023Emergent Correspondence from Image Diffusioncited by 437
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2020Integrated Task and Motion Planningcited by 281
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
- 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
- 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
- 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 130
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
- 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
- 2016Revisiting Active Perceptioncited by 90
- 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
- 2024Language Agents as Optimizable Graphscited by 81
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
- 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
- 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
- 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
- 2020A Survey of Behavior Trees in Robotics and AIcited by 53
- 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
- 2024CleanDIFT: Diffusion Features without Noisecited by 42
- 2024MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chainscited by 25
- 2025MonoDream: Monocular Vision-Language Navigation with Panoramic Dreamingcited by 15
- 2025UniForward: Unified 3D Scene and Semantic Field Reconstruction via Feed-Forward Gaussian Splatting from Only Sparse-View Imagescited by 13
- 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harnesscited by 6
- 2026MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLNcited by 1
Built on by
Not cited yet.