# Diagnosing the Environment Bias in Vision-and-Language Navigation

arXiv: https://arxiv.org/abs/2005.03086 · HTML: https://mlsanity.com/paper/2005.03086/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2020.

## Built on by (most cited first, 30 of 41)

- [History Aware Multimodal Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2110.13309/) (2021)
- [Airbert: In-domain Pretraining for Vision-and-Language Navigation](https://mlsanity.com/paper/2108.09105/) (2021)
- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [Scaling Data Generation in Vision-and-Language Navigation](https://mlsanity.com/paper/2307.15644/) (2023)
- [EnvEdit: Environment Editing for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.15685/) (2022)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning](https://mlsanity.com/paper/2210.03112/) (2022)
- [Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring](https://mlsanity.com/paper/2106.03427/) (2021)
- [SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2110.14143/) (2021)
- [Vision-and-Language Navigation via Causal Learning](https://mlsanity.com/paper/2404.10241/) (2024)
- [Simple and Effective Synthesis of Indoor 3D Scenes](https://mlsanity.com/paper/2204.02960/) (2022)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2106.13948/) (2021)
- [Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation](https://mlsanity.com/paper/2308.12587/) (2023)
- [Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas](https://mlsanity.com/paper/2203.13838/) (2022)
- [Causal Balancing for Domain Generalization](https://mlsanity.com/paper/2206.05263/) (2022)
- [A Dual Semantic-Aware Recurrent Global-Adaptive Network For Vision-and-Language Navigation](https://mlsanity.com/paper/2305.03602/) (2023)
- [VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator](https://mlsanity.com/paper/2105.11589/) (2021)
- [Kefa: A Knowledge Enhanced and Fine-grained Aligned Speaker for Navigation Instruction Generation](https://mlsanity.com/paper/2307.13368/) (2023)
- [Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigation](https://mlsanity.com/paper/2403.10700/) (2024)
- [CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representations](https://mlsanity.com/paper/2207.02185/) (2022)
- [Vision-Language Navigation with Embodied Intelligence: A Survey](https://mlsanity.com/paper/2402.14304/) (2024)
- [Why Only Text: Empowering Vision-and-Language Navigation with Multi-modal Prompts](https://mlsanity.com/paper/2406.02208/) (2024)
- [L2C: Describing Visual Differences Needs Semantic Understanding of Individuals](https://mlsanity.com/paper/2102.01860/) (2021)
- [FedVLN: Privacy-preserving Federated Vision-and-Language Navigation](https://mlsanity.com/paper/2203.14936/) (2022)
- [PASTS: Progress-Aware Spatio-Temporal Transformer Speaker For Vision-and-Language Navigation](https://mlsanity.com/paper/2305.11918/) (2023)
- [Multi-Level Compositional Reasoning for Interactive Instruction Following](https://mlsanity.com/paper/2308.09387/) (2023)
- [Vision-Language Navigation with Continual Learning](https://mlsanity.com/paper/2409.02561/) (2024)
- [NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation](https://mlsanity.com/paper/2510.16457/) (2025)
- [Limited Linguistic Diversity in Embodied AI Datasets](https://mlsanity.com/paper/2601.03136/) (2026)
- [Counterfactual Maximum Likelihood Estimation for Training Deep Networks](https://mlsanity.com/paper/2106.03831/) (2021)

## Abstract (arXiv)

Vision-and-Language Navigation (VLN) requires an agent to follow natural-language instructions, explore the given environments, and reach the desired target locations. These step-by-step navigational instructions are crucial when the agent is navigating new environments about which it has no prior knowledge. Most recent works that study VLN observe a significant performance drop when tested on unseen environments (i.e., environments not used in training), indicating that the neural agent models are highly biased towards training environments. Although this issue is considered as one of the major challenges in VLN research, it is still under-studied and needs a clearer explanation. In this work, we design novel diagnosis experiments via environment re-splitting and feature replacement, looking into possible reasons for this environment bias. We observe that neither the language nor the underlying navigational graph, but the low-level visual appearance conveyed by ResNet features directly affects the agent model and contributes to this environment bias in results. According to this observation, we explore several kinds of semantic representations that contain less low-level visual information, hence the agent learned with these features could be better generalized to unseen testing environments. Without modifying the baseline agent model and its training method, our explored semantic features significantly decrease the performance gaps between seen and unseen on multiple datasets (i.e. R2R, R4R, and CVDN) and achieve competitive unseen results to previous state-of-the-art models. Our code and features are available at: https://github.com/zhangybzbo/EnvBiasVLN