mlsanityData to 7 Oct 2026

Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning

arXiv 2302.03281 ↗First posted 7 Feb 2023cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Modern representation learning methods often struggle to adapt quickly under non-stationarity because they suffer from catastrophic forgetting and decaying plasticity. Such problems prevent learners from fast adaptation since they may forget useful features or have difficulty learning new ones. Hence, these methods are rendered ineffective for continual learning. This paper proposes Utility-based Perturbed Gradient Descent (UPGD), an online learning algorithm well-suited for continual learning agents. UPGD protects useful weights or features from forgetting and perturbs less useful ones based on their utilities. Our empirical results show that UPGD helps reduce forgetting and maintain plasticity, enabling modern representation learning methods to work effectively in continual learning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 29 of its 62 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

29 of 62 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  8. 2018Experience Replay for Continual Learningcited by 1,074
  9. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  10. 2019Importance Estimation for Neural Network Pruningcited by 531
  11. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  12. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
17 more
  1. 2018Selective Experience Replay for Lifelong Learningcited by 239
  2. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  3. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  4. 2018Task-Free Continual Learningcited by 219
  5. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
  6. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 144
  7. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 133
  8. 2020Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamicscited by 93
  9. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  10. 2020Lookahead: A Far-Sighted Alternative of Magnitude-based Pruningcited by 61
  11. 2019Task Agnostic Continual Learning via Meta Learningcited by 53
  12. 2022Anticorrelated Noise Injection for Improved Generalizationcited by 41
  13. 2018The Utility of Sparse Representations for Control in Reinforcement Learningcited by 23
  14. 2020On Catastrophic Interference in Atari 2600 Gamescited by 13
  15. 2019Meta-descent for Online, Continual Predictioncited by 10
  16. 2022HesScale: Scalable Computation of Hessian Diagonalscited by 8
  17. 2019Hebbian-Descentcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 2.

5 papers.

  1. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  2. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  3. 2024Forget Sharpness: Perturbed Forgetting of Model Biases Within SAM Dynamicscited by 0
  4. 2026Beyond Attention: True Adaptive World Models via Spherical Kernel Operatorcited by 0
  5. 2026Normative Loss Landscape Navigation: A Trajectory-Based Approach to Mitigating Forgetting in Incremental Learningcited by 0