mlsanityData to 7 Oct 2026

Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method

arXiv 2412.09082 ↗First posted 12 Dec 2024cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

Existing Vision-Language Navigation (VLN) methods primarily focus on single-stage navigation, limiting their effectiveness in multi-stage and long-horizon tasks within complex and dynamic environments. To address these limitations, we propose a novel VLN task, named Long-Horizon Vision-Language Navigation (LH-VLN), which emphasizes long-term planning and decision consistency across consecutive subtasks. Furthermore, to support LH-VLN, we develop an automated data generation platform NavGen, which constructs datasets with complex task structures and improves data utility through a bidirectional, multi-granularity generation approach. To accurately evaluate complex tasks, we construct the Long-Horizon Planning and Reasoning in VLN (LHPR-VLN) benchmark consisting of 3,260 tasks with an average of 150 task steps, serving as the first dataset specifically designed for the long-horizon vision-language navigation task. Furthermore, we propose Independent Success Rate (ISR), Conditional Success Rate (CSR), and CSR weight by Ground Truth (CGT) metrics, to provide fine-grained assessments of task completion. To improve model adaptability in complex tasks, we propose a novel Multi-Granularity Dynamic Memory (MGDM) module that integrates short-term memory blurring with long-term memory retrieval to enable flexible navigation in dynamic environments. Our platform, benchmark and method supply LH-VLN with a robust data generation pipeline, comprehensive model evaluation dataset, reasonable metrics, and a novel VLN model, establishing a foundational framework for advancing LH-VLN.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 44 of its 54 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

44 of 54 references.

References · 44

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  7. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  8. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  9. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  10. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  11. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  12. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
32 more
  1. 2019Vision-and-Dialog Navigationcited by 246
  2. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  3. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  4. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  5. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  6. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  7. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  8. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  9. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  10. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  11. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  12. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  13. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 124
  14. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  15. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  16. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  17. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
  18. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  19. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  20. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  21. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 92
  22. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 87
  23. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  24. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  25. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  26. 2024MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D Worldcited by 38
  27. 2022Iterative Vision-and-Language Navigationcited by 34
  28. 2024RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Modelscited by 20
  29. 2018Visual Memory for Robust Path Followingcited by 16
  30. 2023Visual Causal Scene Refinement for Video Question Answeringcited by 16
  31. 2024InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interactioncited by 13
  32. 2024Embodied Instruction Following in Unknown Environmentscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 28; 2026: 36.

66 papers.

Sort:

By area:

  1. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  2. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 43
  3. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  4. 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learningcited by 30
  5. 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
  6. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  7. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  8. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  9. 2025Towards Physically Executable 3D Gaussian for Embodied Navigationcited by 16
  10. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  11. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 12
  12. 2025IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environmentscited by 9
38 more
  1. 2024Modality-Inconsistent Continual Learning of Multimodal Large Language Modelscited by 6
  2. 2025HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactionscited by 6
  3. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigationcited by 6
  4. 2026Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigationcited by 6
  5. 2025Mobile-Agent-RAG: Driving Smart Multi-Agent Coordination with Contextual Knowledge Empowerment for Long-Horizon Mobile Automationcited by 5
  6. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Explorationcited by 5
  7. 2025D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigationcited by 5
  8. 2025DeliveryBench: Can Agents Earn Profit in Real World?cited by 5
  9. 2025Breakpoint: Scalable evaluation of system-level reasoning in LLM code agentscited by 4
  10. 2025Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotationscited by 4
  11. 2025AgriVLN: Vision-and-Language Navigation for Agricultural Robotscited by 4
  12. 2025OVSeg3R: Learn Open-vocabulary Instance Segmentation from 2D via 3D Reconstructioncited by 4
  13. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  14. 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigationcited by 3
  15. 2025UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Modelcited by 3
  16. 2026SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planningcited by 3
  17. 2026LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigationcited by 3
  18. 2026World Models as Group Actionscited by 3
  19. 2025ToolScope: An Agentic Framework for Vision-Guided and Long-Horizon Tool Usecited by 2
  20. 2026From Instruction to Event: Sound-Triggered Mobile Manipulationcited by 2
  21. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigationcited by 2
  22. 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Taskscited by 2
  23. 2026Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigationcited by 2
  24. 2025SUM-AgriVLN: Spatial Understanding Memory for Agricultural Vision-and-Language Navigationcited by 1
  25. 2026All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptationcited by 1
  26. 2026Exploring the Use of VLMs for Navigation Assistance for People with Blindness and Low Visioncited by 1
  27. 2026Novel Memory Forgetting Techniques for Autonomous AI Agents: Balancing Relevance and Efficiencycited by 1
  28. 2026Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigationcited by 1
  29. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  30. 2024VisionGRU: A Linear-Complexity RNN Model for Efficient Image Analysiscited by 0
  31. 2025Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencingcited by 0
  32. 2025OceanGym: A Benchmark Environment for Underwater Embodied Agentscited by 0
  33. 2025TP-MDDN: Task-Preferenced Multi-Demand-Driven Navigation with Autonomous Decision-Makingcited by 0
  34. 2026VLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operatorcited by 0
  35. 2026State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Modelscited by 0
  36. 2026CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigationcited by 0
  37. 2026Listening with the Eyes: Benchmarking Egocentric Co-Speech Grounding across Space and Timecited by 0
  38. 2026How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspacecited by 0

50 most cited of 66 All 66 (JSON)