# A Brief Look at Generalization in Visual Meta-Reinforcement Learning

arXiv: https://arxiv.org/abs/2006.07262 · HTML: https://mlsanity.com/paper/2006.07262/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2312.12044/) (2023)
- [On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning](https://mlsanity.com/paper/2206.03271/) (2022)
- [What is Going on Inside Recurrent Meta Reinforcement Learning Agents?](https://mlsanity.com/paper/2104.14644/) (2021)
- [Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learning](https://mlsanity.com/paper/2301.10119/) (2023)
- [Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agents](https://mlsanity.com/paper/2204.06550/) (2022)

## Abstract (arXiv)

Due to the realization that deep reinforcement learning algorithms trained on high-dimensional tasks can strongly overfit to their training environments, there have been several studies that investigated the generalization performance of these algorithms. However, there has been no similar study that evaluated the generalization performance of algorithms that were specifically designed for generalization, i.e. meta-reinforcement learning algorithms. In this paper, we assess the generalization performance of these algorithms by leveraging high-dimensional, procedurally generated environments. We find that these algorithms can display strong overfitting when they are evaluated on challenging tasks. We also observe that scalability to high-dimensional tasks with sparse rewards remains a significant problem among many of the current meta-reinforcement learning algorithms. With these results, we highlight the need for developing meta-reinforcement learning algorithms that can both generalize and scale.