mlsanityData to 7 Oct 2026

Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation

arXiv 2404.01943 ↗First posted 2 Apr 2024cs.CV cs.ROcited by 63

Not on the map. Why?

Abstract (arXiv)

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. At each navigation step, the agent selects from possible candidate locations and then makes the move. For better navigation planning, the lookahead exploration strategy aims to effectively evaluate the agent's next action by accurately anticipating the future environment of candidate locations. To this end, some existing works predict RGB images for future environments, while this strategy suffers from image distortion and high computational cost. To address these issues, we propose the pre-trained hierarchical neural radiance representation model (HNR) to produce multi-level semantic features for future environments, which are more robust and efficient than pixel-wise RGB reconstruction. Furthermore, with the predicted future environmental representations, our lookahead VLN model is able to construct the navigable future path tree and select the optimal path via efficient parallel evaluation. Extensive experiments on the VLN-CE datasets confirm the effectiveness of our method.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

45 of 53 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  10. 2022Visual Language Maps for Robot Navigationcited by 362
  11. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  12. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
33 more
  1. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  2. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2019Vision-and-Dialog Navigationcited by 246
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  6. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  7. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  8. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  9. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  10. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  11. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  12. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  13. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  14. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  15. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  16. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  17. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  18. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  19. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  20. 2019Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigationcited by 81
  21. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  22. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  23. 2020Active Visual Information Gathering for Vision-Language Navigationcited by 55
  24. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
  25. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  26. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  27. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 38
  28. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  29. 20221st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)cited by 30
  30. 2021GEASI: Geodesic-based Earliest Activation Sites Identification in cardiac modelscited by 9
  31. 2022ULN: Towards Underspecified Vision-and-Language Navigationcited by 5
  32. 2023Language-enhanced RNR-Map: Querying Renderable Neural Radiance Field maps with natural languagecited by 5
  33. 2018Model Learning for Look-ahead Exploration in Continuous Controlcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 25; 2026: 28.

63 papers.

Sort:

By area:

  1. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  3. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  4. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  5. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  6. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  7. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  8. 2025CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memorycited by 30
  9. 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 26
  10. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
  11. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  12. 2024Vision-Language Navigation with Energy-Based Policycited by 19
38 more
  1. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  2. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  3. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  4. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  5. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 16
  6. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  7. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  8. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  9. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 12
  10. 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 11
  11. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  12. 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 10
  13. 2026SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigationcited by 8
  14. 2026Dual-Anchoring: Addressing State Drift in Vision-Language Navigationcited by 7
  15. 2025View Invariant Learning for Vision-Language Navigation in Continuous Environmentscited by 6
  16. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6
  17. 2025STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimizationcited by 6
  18. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigationcited by 6
  19. 2026textscNaVIDA: Vision-Language Navigation with Inverse Dynamics Augmentationcited by 5
  20. 2026One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representationcited by 5
  21. 2026ABot-N1: Toward a General Visual Language Navigation Foundation Modelcited by 5
  22. 2025Perspective from a Higher Dimension: Can 3D Geometric Priors Help Visual Floorplan Localization?cited by 4
  23. 2025DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigationcited by 4
  24. 20263D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigationcited by 4
  25. 2025Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory Systemcited by 3
  26. 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigationcited by 3
  27. 2025UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Modelcited by 3
  28. 2026SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planningcited by 3
  29. 2026Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environmentscited by 3
  30. 2026Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detectioncited by 3
  31. 2024World-Consistent Data Generation for Vision-and-Language Navigationcited by 2
  32. 2025Hierarchical Scoring with 3D Gaussian Splatting for Instance Image-Goal Navigationcited by 2
  33. 2026Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigationcited by 2
  34. 2024AdaVLN: Towards Visual Language Navigation in Continuous Indoor Environments with Moving Humanscited by 1
  35. 2026TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigationcited by 1
  36. 2026SpikeVLA: Vision-Language-Action Models with Spiking Neural Networkscited by 1
  37. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  38. 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1

50 most cited of 63 All 63 (JSON)