mlsanityData to 7 Oct 2026

Measuring and Mitigating Interference in Reinforcement Learning

arXiv 2307.04887 ↗First posted 10 Jul 2023cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it. Before overcoming interference we must understand it better. In this work, we provide a definition and novel measure of interference for value-based reinforcement learning methods such as Fitted Q-Iteration and DQN. We systematically evaluate our measure of interference, showing that it correlates with instability in control performance, across a variety of network architectures. Our new interference measure allows us to ask novel scientific questions about commonly used deep learning architectures and study learning algorithms which mitigate interference. Lastly, we outline a class of algorithms which we call online-aware that are designed to mitigate interference, and show they do reduce interference according to our measure and that they improve stability and performance in several classic control environments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

31 of 45 references.

References · 31

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  8. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  9. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  10. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  11. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  12. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
19 more
  1. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  2. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  3. 2019Meta-Learning Representations for Continual Learningcited by 189
  4. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  5. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  6. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  7. 2018Generalization and Regularization in DQNcited by 117
  8. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  9. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  10. 2019Toward Understanding Catastrophic Forgetting in Continual Learningcited by 67
  11. 2017Towards Generalization and Simplicity in Continuous Controlcited by 65
  12. 2019Measuring the Reliability of Reinforcement Learning Algorithmscited by 49
  13. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  14. 2019Towards Characterizing Divergence in Deep Q-Learningcited by 43
  15. 2020Interference and Generalization in Temporal Difference Learningcited by 41
  16. 2019Ray Interference: a Source of Plateaus in Deep Reinforcement Learningcited by 40
  17. 2022The Phenomenon of Policy Churncited by 28
  18. 2018The Utility of Sparse Representations for Control in Reinforcement Learningcited by 23
  19. 2020On Catastrophic Interference in Atari 2600 Gamescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  2. 2024Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churncited by 16
  3. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  4. 2026Value Bonuses using Ensemble Errors for Exploration in Reinforcement Learningcited by 1
  5. 2026Overcoming Valid Action Suppression in Unmasked Policy Gradient Algorithmscited by 0