# Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation

arXiv: https://arxiv.org/abs/2404.01943 · HTML: https://mlsanity.com/paper/2404.01943/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 53 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments](https://mlsanity.com/paper/1904.10151/) (2019)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [A Recurrent Vision-and-Language BERT for Navigation](https://mlsanity.com/paper/2011.13922/) (2020)
- [History Aware Multimodal Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2110.13309/) (2021)
- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout](https://mlsanity.com/paper/1904.04195/) (2019)
- [Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2202.11742/) (2022)
- [ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2304.03047/) (2023)
- [Episodic Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2105.06453/) (2021)
- [Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.02764/) (2022)
- [Unconstrained Scene Generation with Locally Conditioned Radiance Fields](https://mlsanity.com/paper/2104.00670/) (2021)
- [Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation](https://mlsanity.com/paper/2308.07931/) (2023)
- [GridMM: Grid Memory Map for Vision-and-Language Navigation](https://mlsanity.com/paper/2307.12907/) (2023)
- [BEVBert: Multimodal Map Pre-training for Language-guided Navigation](https://mlsanity.com/paper/2212.04385/) (2022)
- [SOON: Scenario Oriented Object Navigation with Graph-based Exploration](https://mlsanity.com/paper/2103.17138/) (2021)
- [Pathdreamer: A World Model for Indoor Navigation](https://mlsanity.com/paper/2105.08756/) (2021)
- [Scaling Data Generation in Vision-and-Language Navigation](https://mlsanity.com/paper/2307.15644/) (2023)
- [Cross-modal Map Learning for Vision and Language Navigation](https://mlsanity.com/paper/2203.05137/) (2022)
- [Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2204.09667/) (2022)

## Built on by (most cited first, 30 of 65)

- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling](https://mlsanity.com/paper/2507.05240/) (2025)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method](https://mlsanity.com/paper/2412.09082/) (2024)
- [NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2506.23468/) (2025)
- [Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation](https://mlsanity.com/paper/2406.09798/) (2024)
- [CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory](https://mlsanity.com/paper/2505.05622/) (2025)
- [CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Model](https://mlsanity.com/paper/2508.10416/) (2025)
- [ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigation](https://mlsanity.com/paper/2602.11598/) (2026)
- [Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation](https://mlsanity.com/paper/2505.11383/) (2025)
- [g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Tasks](https://mlsanity.com/paper/2411.17030/) (2024)
- [Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel](https://mlsanity.com/paper/2412.08467/) (2024)
- [Vision-Language Navigation with Energy-Based Policy](https://mlsanity.com/paper/2410.14250/) (2024)
- [VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation](https://mlsanity.com/paper/2512.19021/) (2025)
- [Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation](https://mlsanity.com/paper/2505.20897/) (2025)
- [AstraNav-World: World Model for Foresight Control and Consistency](https://mlsanity.com/paper/2512.21714/) (2025)
- [NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction](https://mlsanity.com/paper/2512.01550/) (2025)
- [COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigation](https://mlsanity.com/paper/2503.24065/) (2025)
- [MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots](https://mlsanity.com/paper/2511.17889/) (2025)
- [Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Construction](https://mlsanity.com/paper/2503.11091/) (2025)
- [Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space](https://mlsanity.com/paper/2512.21887/) (2025)
- [Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System](https://mlsanity.com/paper/2606.18112/) (2026)
- [GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigation](https://mlsanity.com/paper/2509.10454/) (2025)
- [AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation](https://mlsanity.com/paper/2605.22816/) (2026)
- [SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigation](https://mlsanity.com/paper/2601.06806/) (2026)
- [Dual-Anchoring: Addressing State Drift in Vision-Language Navigation](https://mlsanity.com/paper/2604.17473/) (2026)
- [View Invariant Learning for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2507.08831/) (2025)
- [NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation](https://mlsanity.com/paper/2510.16457/) (2025)
- [STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimization](https://mlsanity.com/paper/2511.00033/) (2025)

## Abstract (arXiv)

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. At each navigation step, the agent selects from possible candidate locations and then makes the move. For better navigation planning, the lookahead exploration strategy aims to effectively evaluate the agent's next action by accurately anticipating the future environment of candidate locations. To this end, some existing works predict RGB images for future environments, while this strategy suffers from image distortion and high computational cost. To address these issues, we propose the pre-trained hierarchical neural radiance representation model (HNR) to produce multi-level semantic features for future environments, which are more robust and efficient than pixel-wise RGB reconstruction. Furthermore, with the predicted future environmental representations, our lookahead VLN model is able to construct the navigable future path tree and select the optimal path via efficient parallel evaluation. Extensive experiments on the VLN-CE datasets confirm the effectiveness of our method.