mlsanityData to 7 Oct 2026

Continual Backprop: Stochastic Gradient Descent with Persistent Randomness

arXiv 2108.06325 ↗First posted 13 Aug 2021cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

The Backprop algorithm for learning in neural networks utilizes two mechanisms: first, stochastic gradient descent and second, initialization with small random weights, where the latter is essential to the effectiveness of the former. We show that in continual learning setups, Backprop performs well initially, but over time its performance degrades. Stochastic gradient descent alone is insufficient to learn continually; the initial randomness enables only initial learning but not continual learning. To the best of our knowledge, ours is the first result showing this degradation in Backprop's ability to learn. To address this degradation in Backprop's plasticity, we propose an algorithm that continually injects random features alongside gradient descent using a new generate-and-test process. We call this the textitContinual Backprop algorithm. We show that, unlike Backprop, Continual Backprop is able to continually adapt in both supervised and reinforcement learning (RL) problems. Continual Backprop has the same computational complexity as Backprop and can be seen as a natural extension of Backprop for continual learning.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 23 of its 53 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

23 of 53 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  10. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  11. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  12. 2019Gradient based sample selection for online continual learningcited by 568
11 more
  1. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  2. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  3. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  4. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  5. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  6. 2018Online Learning: A Comprehensive Surveycited by 203
  7. 2019Meta-Learning Representations for Continual Learningcited by 189
  8. 2017Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environmentscited by 106
  9. 2017Online Deep Learning: Learning Deep Neural Networks on the Flycited by 86
  10. 2019Growing a Brain: Fine-Tuning by Increasing Model Capacitycited by 43
  11. 2019Online Normalization for Training Neural Networkscited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 20; 2024: 25; 2025: 13; 2026: 15.

79 papers.

Sort:

By area:

  1. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  2. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  3. 2023Understanding plasticity in neural networkscited by 170
  4. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  5. 2023A Definition of Continual Reinforcement Learningcited by 98
  6. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  7. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  8. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  9. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 63
  10. 2024Normalization and effective learning rates in reinforcement learningcited by 55
  11. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  12. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
38 more
  1. 2023Meta-Learning Online Adaptation of Language Modelscited by 43
  2. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  3. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  4. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  5. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 35
  6. 2023Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learningcited by 34
  7. 2024Learning Continually by Spectral Regularizationcited by 34
  8. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  9. 2021Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Timecited by 27
  10. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  11. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  12. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  13. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  14. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  15. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  16. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  17. 2024Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Resetcited by 17
  18. 2025Optimizers Qualitatively Alter Solutions And We Should Leverage Thiscited by 17
  19. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 16
  20. 2024Plastic Learning with Deep Fourier Featurescited by 14
  21. 2023Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learningcited by 13
  22. 2022Continual task learning in natural and artificial agentscited by 12
  23. 2024Self-Normalized Resets for Plasticity in Continual Learningcited by 11
  24. 2025Plasticity as the Mirror of Empowermentcited by 11
  25. 2024Position: Lifetime tuning is incompatible with continual reinforcement learningcited by 9
  26. 2024Remove Symmetries to Control Model Expressivity and Improve Optimizationcited by 8
  27. 2025On-device edge learning for IoT data streams: a surveycited by 8
  28. 2024DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticitycited by 7
  29. 2025What Can Grokking Teach Us About Learning Under Nonstationarity?cited by 7
  30. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 7
  31. 2023Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learningcited by 6
  32. 2025Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Losscited by 6
  33. 2025Reinitializing weights vs units for maintaining plasticity in neural networkscited by 6
  34. 2025Activation Function Design Sustains Plasticity in Continual Learningcited by 6
  35. 2025Bridging Streaming Continual Learning via In-Context Large Tabular Modelscited by 6
  36. 2023Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learningcited by 5
  37. 2023Measuring and Mitigating Interference in Reinforcement Learningcited by 5
  38. 2024Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neuronscited by 5

50 most cited of 79 All 79 (JSON)