# Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agents

arXiv: https://arxiv.org/abs/2204.06550 · HTML: https://mlsanity.com/paper/2204.06550/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Assessing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.12282/) (2018)
- [Generalization and Regularization in DQN](https://mlsanity.com/paper/1810.00123/) (2018)
- [Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck](https://mlsanity.com/paper/1910.12911/) (2019)
- [Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation](https://mlsanity.com/paper/1806.10729/) (2018)
- [Gotta Learn Fast: A New Benchmark for Generalization in RL](https://mlsanity.com/paper/1804.03720/) (2018)
- [Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting](https://mlsanity.com/paper/1812.02464/) (2018)
- [Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning](https://mlsanity.com/paper/2006.01096/) (2020)
- [Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.00614/) (2020)
- [Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks](https://mlsanity.com/paper/2003.07417/) (2020)
- [Reinforcement Learning Generalization with Surprise Minimization](https://mlsanity.com/paper/2004.12399/) (2020)
- [A Brief Look at Generalization in Visual Meta-Reinforcement Learning](https://mlsanity.com/paper/2006.07262/) (2020)

## Abstract (arXiv)

Adapting a Reinforcement Learning (RL) agent to an unseen environment is a difficult task due to typical over-fitting on the training environment. RL agents are often capable of solving environments very close to the trained environment, but when environments become substantially different, their performance quickly drops. When agents are retrained on new environments, a second issue arises: there is a risk of catastrophic forgetting, where the performance on previously seen environments is seriously hampered. This paper proposes a novel approach that exploits an eco-system of agents to address both concerns. Hereby, the (limited) adaptive power of individual agents is harvested to build a highly adaptive eco-system.