# Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks

arXiv: https://arxiv.org/abs/2003.07417 · HTML: https://mlsanity.com/paper/2003.07417/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [Streaming Deep Reinforcement Learning Finally Works](https://mlsanity.com/paper/2410.14606/) (2024)
- [Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn](https://mlsanity.com/paper/2409.04792/) (2024)
- [Behavior Alignment via Reward Function Optimization](https://mlsanity.com/paper/2310.19007/) (2023)
- [Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation](https://mlsanity.com/paper/2109.00525/) (2021)
- [Learning Dynamics and Generalization in Reinforcement Learning](https://mlsanity.com/paper/2206.02126/) (2022)
- [Reinforcement Learning with Automated Auxiliary Loss Search](https://mlsanity.com/paper/2210.06041/) (2022)
- [Harnessing Discrete Representations For Continual Reinforcement Learning](https://mlsanity.com/paper/2312.01203/) (2023)
- [Elephant Neural Networks: Born to Be a Continual Learner](https://mlsanity.com/paper/2310.01365/) (2023)
- [Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidation](https://mlsanity.com/paper/2205.10868/) (2022)
- [Does the Adam Optimizer Exacerbate Catastrophic Forgetting?](https://mlsanity.com/paper/2102.07686/) (2021)
- [Frequency and Generalisation of Periodic Activation Functions in Reinforcement Learning](https://mlsanity.com/paper/2407.06756/) (2024)
- [LiBOG: Lifelong Learning for Black-Box Optimizer Generation](https://mlsanity.com/paper/2505.13025/) (2025)
- [Is Exploration or Optimization the Problem for Deep Reinforcement Learning?](https://mlsanity.com/paper/2508.01329/) (2025)
- [Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agents](https://mlsanity.com/paper/2204.06550/) (2022)
- [Generalization Through the Lens of Learning Dynamics](https://mlsanity.com/paper/2212.05377/) (2022)
- [Improving generalization in reinforcement learning through forked agents](https://mlsanity.com/paper/2212.06451/) (2022)
- [Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functions](https://mlsanity.com/paper/2509.19159/) (2025)

## Abstract (arXiv)

Reinforcement learning systems require good representations to work well. For decades practical success in reinforcement learning was limited to small domains. Deep reinforcement learning systems, on the other hand, are scalable, not dependent on domain specific prior knowledge and have been successfully used to play Atari, in 3D navigation from pixels, and to control high degree of freedom robots. Unfortunately, the performance of deep reinforcement learning systems is sensitive to hyper-parameter settings and architecture choices. Even well tuned systems exhibit significant instability both within a trial and across experiment replications. In practice, significant expertise and trial and error are usually required to achieve good performance. One potential source of the problem is known as catastrophic interference: when later training decreases performance by overriding previous learning. Interestingly, the powerful generalization that makes Neural Networks (NN) so effective in batch supervised learning might explain the challenges when applying them in reinforcement learning tasks. In this paper, we explore how online NN training and interference interact in reinforcement learning. We find that simply re-mapping the input observations to a high-dimensional space improves learning speed and parameter sensitivity. We also show this preprocessing reduces interference in prediction tasks. More practically, we provide a simple approach to NN training that is easy to implement, and requires little additional computation. We demonstrate that our approach improves performance in both prediction and control with an extensive batch of experiments in classic control domains.