mlsanityData to 7 Oct 2026

Self-Activating Neural Ensembles for Continual Reinforcement Learning

arXiv 2301.00141 ↗First posted 31 Dec 2022cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent's next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 70 of its 104 references that are in the map. First posted on arXiv 31 Dec 2022.

Builds on

70 of 104 references.

References · 70

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2016Learning without Forgettingcited by 2,465
  7. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  9. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  10. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  11. 2016Progressive Neural Networkscited by 1,332
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
38 more
  1. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  2. 2017Continual Learning with Deep Generative Replaycited by 1,152
  3. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  4. 2018Experience Replay for Continual Learningcited by 1,074
  5. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  6. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  9. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  10. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  11. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  12. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  13. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  14. 2015Policy Distillationcited by 414
  15. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  16. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 369
  17. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  18. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  19. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  20. 2019Self-Supervised Exploration via Disagreementcited by 263
  21. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  22. 2018Selective Experience Replay for Lifelong Learningcited by 239
  23. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  24. 2018Task-Free Continual Learningcited by 219
  25. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197
  26. 2018Reinforced Continual Learningcited by 186
  27. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  28. 2015Training Very Deep Networkscited by 172
  29. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 133
  30. 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
  31. 2017Learning Independent Causal Mechanismscited by 104
  32. 2018Generative replay with feedback connections as a general strategy for continual learningcited by 103
  33. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  34. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 86
  35. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  36. 2018Generative Models from the perspective of Continual Learningcited by 69
  37. 2019Superposition of many models into onecited by 64
  38. 2018Task Agnostic Continual Learning Using Online Variational Bayescited by 61

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 2.

5 papers.

  1. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  2. 2025A Survey of Continual Reinforcement Learningcited by 16
  3. 2023Evaluating Continual Learning on a Home Robotcited by 4
  4. 2024Continual learning with the neural tangent ensemblecited by 4
  5. 2025Lifelong Reinforcement Learning with Similarity-Driven Weighting by Large Modelscited by 0