# Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation

arXiv: https://arxiv.org/abs/1911.07450 · HTML: https://mlsanity.com/paper/1911.07450/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 30 of 40)

- [Episodic Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2105.06453/) (2021)
- [Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions](https://mlsanity.com/paper/2308.04152/) (2023)
- [Fine-Grained Semantically Aligned Vision-Language Pre-Training](https://mlsanity.com/paper/2208.02515/) (2022)
- [Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation](https://mlsanity.com/paper/2202.02440/) (2022)
- [Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning](https://mlsanity.com/paper/2203.13049/) (2022)
- [BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation](https://mlsanity.com/paper/2203.02533/) (2022)
- [Hierarchical Object-to-Zone Graph for Object Navigation](https://mlsanity.com/paper/2109.02066/) (2021)
- [MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning](https://mlsanity.com/paper/2112.06558/) (2021)
- [One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones](https://mlsanity.com/paper/2202.07028/) (2022)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Interactron: Embodied Adaptive Object Detection](https://mlsanity.com/paper/2202.00660/) (2022)
- [Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models](https://mlsanity.com/paper/2303.06571/) (2023)
- [End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding](https://mlsanity.com/paper/2203.08013/) (2022)
- [Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding](https://mlsanity.com/paper/2301.09071/) (2023)
- [Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference](https://mlsanity.com/paper/2107.12270/) (2021)
- [Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation](https://mlsanity.com/paper/2105.09637/) (2021)
- [Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels](https://mlsanity.com/paper/2304.10539/) (2023)
- [Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization](https://mlsanity.com/paper/2303.12314/) (2023)
- [Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge](https://mlsanity.com/paper/2109.09531/) (2021)
- [Why Do We Click: Visual Impression-aware News Recommendation](https://mlsanity.com/paper/2109.12651/) (2021)
- [Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos](https://mlsanity.com/paper/2208.01954/) (2022)
- [Boosting Virtual Agent Learning and Reasoning: A Step-Wise, Multi-Dimensional, and Generalist Reward Model with Benchmark](https://mlsanity.com/paper/2503.18665/) (2025)
- [What do navigation agents learn about their environment?](https://mlsanity.com/paper/2206.08500/) (2022)
- [EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model](https://mlsanity.com/paper/2210.00498/) (2022)
- [What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities](https://mlsanity.com/paper/2506.08933/) (2025)
- [Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile](https://mlsanity.com/paper/2206.01944/) (2022)
- [I3: Intent-Introspective Retrieval Conditioned on Instructions](https://mlsanity.com/paper/2308.10025/) (2023)
- [VLSNR:Vision-Linguistics Coordination Time Sequence-aware News Recommendation](https://mlsanity.com/paper/2210.02946/) (2022)
- [Human Simulation Computation: A Human-Inspired Framework for Adaptive AI Systems](https://mlsanity.com/paper/2601.13887/) (2026)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)

## Abstract (arXiv)

Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e.g., television) using only visual observations. A key challenge for current deep reinforcement learning models lies in the requirements for a large amount of training data. It is exceedingly expensive to construct sufficient 3D synthetic environments annotated with the target object information. In this paper, we focus on visual navigation in the low-resource setting, where we have only a few training environments annotated with object information. We propose a novel unsupervised reinforcement learning approach to learn transferable meta-skills (e.g., bypass obstacles, go straight) from unannotated environments without any supervisory signals. The agent can then fast adapt to visual navigation through learning a high-level master policy to combine these meta-skills, when the visual-navigation-specified reward is provided. Evaluation in the AI2-THOR environments shows that our method significantly outperforms the baseline by 53.34% relatively on SPL, and further qualitative analysis demonstrates that our method learns transferable motor primitives for visual navigation.