# Visual Perception Generalization for Vision-and-Language Navigation via Meta-Learning

arXiv: https://arxiv.org/abs/2012.05446 · HTML: https://mlsanity.com/paper/2012.05446/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 7 of 7)

- [Vision-Language Navigation: A Survey and Taxonomy](https://mlsanity.com/paper/2108.11544/) (2021)
- [Interactron: Embodied Adaptive Object Detection](https://mlsanity.com/paper/2202.00660/) (2022)
- [Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidance](https://mlsanity.com/paper/2309.01448/) (2023)
- [Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2301.04352/) (2023)
- [Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation](https://mlsanity.com/paper/2503.18065/) (2025)
- [MetaLDC: Meta Learning of Low-Dimensional Computing Classifiers for Fast On-Device Adaption](https://mlsanity.com/paper/2302.12347/) (2023)
- [Causality-based Cross-Modal Representation Learning for Vision-and-Language Navigation](https://mlsanity.com/paper/2403.03405/) (2024)

## Abstract (arXiv)

Vision-and-language navigation (VLN) is a challenging task that requires an agent to navigate in real-world environments by understanding natural language instructions and visual information received in real-time. Prior works have implemented VLN tasks on continuous environments or physical robots, all of which use a fixed camera configuration due to the limitations of datasets, such as 1.5 meters height, 90 degrees horizontal field of view (HFOV), etc. However, real-life robots with different purposes have multiple camera configurations, and the huge gap in visual information makes it difficult to directly transfer the learned navigation model between various robots. In this paper, we propose a visual perception generalization strategy based on meta-learning, which enables the agent to fast adapt to a new camera configuration with a few shots. In the training phase, we first locate the generalization problem to the visual perception module, and then compare two meta-learning algorithms for better generalization in seen and unseen environments. One of them uses the Model-Agnostic Meta-Learning (MAML) algorithm that requires a few shot adaptation, and the other refers to a metric-based meta-learning method with a feature-wise affine transformation layer. The experiment results show that our strategy successfully adapts the learned navigation model to a new camera configuration, and the two algorithms show their advantages in seen and unseen environments respectively.