# General Scene Adaptation for Vision-and-Language Navigation

arXiv: https://arxiv.org/abs/2501.17403 · HTML: https://mlsanity.com/paper/2501.17403/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2025.

## Built on by (most cited first, 17 of 17)

- [UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents](https://mlsanity.com/paper/2508.00288/) (2025)
- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation](https://mlsanity.com/paper/2512.19021/) (2025)
- [MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation](https://mlsanity.com/paper/2511.10376/) (2025)
- [NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation](https://mlsanity.com/paper/2510.16457/) (2025)
- [Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation](https://mlsanity.com/paper/2510.08553/) (2025)
- [NoReGeo: Non-Reasoning Geometry Benchmark](https://mlsanity.com/paper/2601.10254/) (2026)
- [Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering](https://mlsanity.com/paper/2510.01483/) (2025)
- [User-Feedback-Driven Adaptation for Vision-and-Language Navigation](https://mlsanity.com/paper/2512.10322/) (2025)
- [Towards Open Environments and Instructions: General Vision-Language Navigation via Fast-Slow Interactive Reasoning](https://mlsanity.com/paper/2601.09111/) (2026)
- [120 Minutes and a Laptop: Minimalist Image-goal Navigation via Unsupervised Exploration and Offline RL](https://mlsanity.com/paper/2603.26441/) (2026)
- [FlowDec: Temporal Conditional Flow Decorruptor for Robust Continuous Vision-Language Navigation](https://mlsanity.com/paper/2606.22424/) (2026)
- [CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation](https://mlsanity.com/paper/2607.02222/) (2026)
- [Goal-oriented Navigation Instruction Generation with Tour Video Priors](https://mlsanity.com/paper/2608.08596/) (2026)
- [FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation](https://mlsanity.com/paper/2609.32855/) (2026)
- [VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience](https://mlsanity.com/paper/2609.34687/) (2026)
- [AVERT-VLN: Abstention-aware Visual Error Recovery and Training for Vision-and-Language Navigation](https://mlsanity.com/paper/2609.39579/) (2026)

## Abstract (arXiv)

Vision-and-Language Navigation (VLN) tasks mainly evaluate agents based on one-time execution of individual instructions across multiple environments, aiming to develop agents capable of functioning in any environment in a zero-shot manner. However, real-world navigation robots often operate in persistent environments with relatively consistent physical layouts, visual observations, and language styles from instructors. Such a gap in the task setting presents an opportunity to improve VLN agents by incorporating continuous adaptation to specific environments. To better reflect these real-world conditions, we introduce GSA-VLN, a novel task requiring agents to execute navigation instructions within a specific scene and simultaneously adapt to it for improved performance over time. To evaluate the proposed task, one has to address two challenges in existing VLN datasets: the lack of OOD data, and the limited number and style diversity of instructions for each scene. Therefore, we propose a new dataset, GSA-R2R, which significantly expands the diversity and quantity of environments and instructions for the R2R dataset to evaluate agent adaptability in both ID and OOD contexts. Furthermore, we design a three-stage instruction orchestration pipeline that leverages LLMs to refine speaker-generated instructions and apply role-playing techniques to rephrase instructions into different speaking styles. This is motivated by the observation that each individual user often has consistent signatures or preferences in their instructions. We conducted extensive experiments on GSA-R2R to thoroughly evaluate our dataset and benchmark various methods. Based on our findings, we propose a novel method, GR-DUET, which incorporates memory-based navigation graphs with an environment-specific training strategy, achieving state-of-the-art results on all GSA-R2R splits.