mlsanityData to 14 Oct 2026

A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control

arXiv 2507.02712 ↗First posted 3 Jul 2025cs.LGcited by 9

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Deep reinforcement learning for continuous control has recently achieved impressive progress. However, existing methods often suffer from primacy bias, a tendency to overfit early experiences stored in the replay buffer, which limits an RL agent's sample efficiency and generalizability. In contrast, humans are less susceptible to such bias, partly due to infantile amnesia, where the formation of new neurons disrupts early memory traces, leading to the forgetting of initial experiences. Inspired by this dual processes of forgetting and growing in neuroscience, in this paper, we propose Forget and Grow (FoG), a new deep RL algorithm with two mechanisms introduced. First, Experience Replay Decay (ER Decay) "forgetting early experience", which balances memory by gradually reducing the influence of early experiences. Second, Network Expansion, "growing neural capacity", which enhances agents' capability to exploit the patterns of existing data by dynamically adding new parameters during training. Empirical results on four major continuous control benchmarks with more than 40 tasks demonstrate the superior performance of FoG against SoTA existing deep RL algorithms, including BRO, SimBa, and TD-MPC2.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 47 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

33 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning13 Oct
  2. 2024Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL28 May
  3. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control25 May

Other references · 30

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2018DeepMind Control Suitecited by 956
  7. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  8. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  9. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
  10. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  11. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  12. 2020Revisiting Fundamentals of Experience Replaycited by 175
18 more
  1. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  2. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  3. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  4. 2020Maxmin Q-learning: Controlling the Estimation Bias of Q-learningcited by 117
  5. 2017A Deeper Look at Experience Replaycited by 105
  6. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  7. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  8. 2020Reinforcement Learning with Combinatorial Actions: An Application to Vehicle Routingcited by 61
  9. 2020On the role of planning in model-based deep reinforcement learningcited by 57
  10. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  11. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  12. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 47
  13. 2020D2RL: Deep Dense Architectures in Reinforcement Learningcited by 42
  14. 2023Efficient Deep Reinforcement Learning Requires Regulating Overfittingcited by 36
  15. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  16. 2023Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reusecited by 12
  17. 2023Mind the Model, Not the Agent: The Primacy Bias in Model-based RLcited by 9
  18. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 71
  2. 2025XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learningcited by 17
  3. 2025Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraintscited by 4
  4. 2026Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learningcited by 2
  5. 2026The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learningcited by 2
  6. 2026Debiased Model-based Representations for Sample-efficient Continuous Controlcited by 1
  7. 2026HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentationcited by 0
  8. 2026Endpoint Replay: Compressing the Recency Buffer in Deep Reinforcement Learningcited by 0
  9. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Experience Replay for Continual Learning
  2. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay
  3. 2024CUER: Corrected Uniform Experience Replay for Off-Policy Continuous Deep Reinforcement Learning Algorithms
  4. 2025Enhancing Deep Deterministic Policy Gradients on Continuous Control Tasks with Decoupled Prioritized Experience Replay
  5. 2023Continual Visual Reinforcement Learning with A Life-Long World Model
  6. 2025Fisher-Guided Selective Forgetting: Mitigating The Primacy Bias in Deep Reinforcement Learning
  7. 2026From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments
  8. 2026Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning