# On Catastrophic Interference in Atari 2600 Games

arXiv: https://arxiv.org/abs/2002.12499 · HTML: https://mlsanity.com/paper/2002.12499/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 13 of 13)

- [Understanding plasticity in neural networks](https://mlsanity.com/paper/2303.01486/) (2023)
- [Understanding and Preventing Capacity Loss in Reinforcement Learning](https://mlsanity.com/paper/2204.09560/) (2022)
- [Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning](https://mlsanity.com/paper/2404.00781/) (2024)
- [Adversarial Cheap Talk](https://mlsanity.com/paper/2211.11030/) (2022)
- [Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation](https://mlsanity.com/paper/2109.00525/) (2021)
- [Learning Dynamics and Generalization in Reinforcement Learning](https://mlsanity.com/paper/2206.02126/) (2022)
- [All You Need Is Supervised Learning: From Imitation Learning to Meta-RL With Upside Down RL](https://mlsanity.com/paper/2202.11960/) (2022)
- [Does the Adam Optimizer Exacerbate Catastrophic Forgetting?](https://mlsanity.com/paper/2102.07686/) (2021)
- [Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning](https://mlsanity.com/paper/2302.03281/) (2023)
- [Measuring and Mitigating Interference in Reinforcement Learning](https://mlsanity.com/paper/2307.04887/) (2023)
- [Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning](https://mlsanity.com/paper/2405.13978/) (2024)
- [ChatASU: Evoking LLM's Reflexion to Truly Understand Aspect Sentiment in Dialogues](https://mlsanity.com/paper/2403.05326/) (2024)
- [Exploring Natural Language-Based Strategies for Efficient Number Learning in Children through Reinforcement Learning](https://mlsanity.com/paper/2410.08334/) (2024)

## Abstract (arXiv)

Model-free deep reinforcement learning is sample inefficient. One hypothesis -- speculated, but not confirmed -- is that catastrophic interference within an environment inhibits learning. We test this hypothesis through a large-scale empirical study in the Arcade Learning Environment (ALE) and, indeed, find supporting evidence. We show that interference causes performance to plateau; the network cannot train on segments beyond the plateau without degrading the policy used to reach there. By synthetically controlling for interference, we demonstrate performance boosts across architectures, learning algorithms and environments. A more refined analysis shows that learning one segment of a game often increases prediction errors elsewhere. Our study provides a clear empirical link between catastrophic interference and sample efficiency in reinforcement learning.