mlsanityData to 14 Oct 2026

Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functions

arXiv 2509.19159 ↗First posted 23 Sep 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Catastrophic forgetting has remained a significant challenge for efficient reinforcement learning for decades (Ring 1994, Rivest and Precup 2003). While recent works have proposed effective methods to mitigate this issue, they mainly focus on the algorithmic side. Meanwhile, we do not fully understand what architectural properties of neural networks lead to catastrophic forgetting. This study aims to fill this gap by studying the role of activation functions in the training dynamics of neural networks and their impact on catastrophic forgetting in reinforcement learning setup. Our study reveals that, besides sparse representations, the gradient sparsity of activation functions also plays an important role in reducing forgetting. Based on this insight, we propose a new class of activation functions, elephant activation functions, that can generate both sparse outputs and sparse gradients. We show that by simply replacing classical activation functions with elephant activation functions in the neural networks of value-based algorithms, we can significantly improve the resilience of neural networks to catastrophic forgetting, thus making reinforcement learning more sample-efficient and memory-efficient.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 71 of its 94 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

71 of 94 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Streaming Deep Reinforcement Learning Finally Works18 Oct

Other references · 70

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2016Layer Normalizationcited by 5,669
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2016Progressive Neural Networkscited by 1,361
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
38 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  2. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  3. 2018On Lazy Training in Differentiable Programmingcited by 675
  4. 2019Three scenarios for continual learningcited by 614
  5. 2020What is the State of Neural Network Pruning?cited by 583
  6. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  7. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  8. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  9. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  10. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  11. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
  12. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  13. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  14. 2017Noisy Networks for Explorationcited by 266
  15. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  16. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  17. 2018Task-Free Continual Learningcited by 221
  18. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
  19. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  20. 2022Patches Are All You Need?cited by 201
  21. 2013Maxout Networkscited by 187
  22. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 182
  23. 2023Understanding plasticity in neural networkscited by 174
  24. 2018Towards Robust Evaluations of Continual Learningcited by 148
  25. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  26. 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 68
  27. 2018Benchmarking Reinforcement Learning Algorithms on Real-World Robotscited by 65
  28. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  29. 2020The Value-Improvement Path: Towards Better Representations for Reinforcement Learningcited by 60
  30. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  31. 2020SpaceNet: Make Free Space For Continual Learningcited by 53
  32. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  33. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
  34. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  35. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  36. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  37. 2019The Local Elasticity of Neural Networkscited by 37
  38. 2021Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connectorcited by 34

50 most cited of 70 All 70 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Activation Function Design Sustains Plasticity in Continual Learning
  2. 2025Fisher-Guided Selective Forgetting: Mitigating The Primacy Bias in Deep Reinforcement Learning
  3. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration
  4. 2026Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
  5. 2026Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
  6. 2024Learning fast changing slow in spiking neural networks
  7. 2025Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient
  8. 2026When Learning Rates Go Wrong: Early Structural Signals in PPO Actor-Critic