mlsanityData to 7 Oct 2026

Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset

arXiv 2411.04034 ↗First posted 6 Nov 2024cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

25 of 50 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  7. 2019Three scenarios for continual learningcited by 610
  8. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  9. 2017Variational Continual Learningcited by 335
  10. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  11. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  12. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
13 more
  1. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  2. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 92
  3. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  4. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  5. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 63
  6. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
  7. 2013Streaming Variational Bayescited by 46
  8. 2021A study on the plasticity of neural networkscited by 44
  9. 2013Dynamical Models and Tracking Regret in Online Convex Programmingcited by 42
  10. 2021MetaGrad: Adaptation using Multiple Learning Rates in Online Learningcited by 26
  11. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  12. 2023Kalman Filter for Online Classification of Non-Stationary Datacited by 13
  13. 2021Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 6.

17 papers.

Sort:

By area:

  1. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  2. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  3. 2025Forget Forgetting: Continual Learning in a World of Abundant Memorycited by 9
  4. 2024A unifying framework for generalised Bayesian online learning in non-stationary environmentscited by 5
  5. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streamingcited by 4
  6. 2025Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learningcited by 2
  7. 2025Preserving Plasticity in Continual Learning with Adaptive Linearity Injectioncited by 1
  8. 2025Is Exploration or Optimization the Problem for Deep Reinforcement Learning?cited by 1
  9. 2026Plasticity-Enhanced Multi-Agent Mixture of Experts for Dynamic Objective Adaptation in UAVs-Assisted Emergency Communication Networkscited by 1
  10. 2025Staying Alive: Online Neural Network Maintenance and Systemic Driftcited by 0
  11. 2025Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Directioncited by 0
  12. 2025Emotion-Inspired Learning Signals (EILS): A Homeostatic Framework for Adaptive Autonomous Agentscited by 0
5 more
  1. 2026Rethinking Plasticity in Deep Reinforcement Learningcited by 0
  2. 2026Forager: a lightweight testbed for continual learning with partial observability in RLcited by 0
  3. 2026Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularizationcited by 0
  4. 2026PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networkscited by 0
  5. 2026Online Learning via Learned Latent Bayesian Trackingcited by 0