mlsanityData to 7 Oct 2026

In value-based deep reinforcement learning, a pruned network is a good network

arXiv 2402.12479 ↗First posted 19 Feb 2024cs.LG cs.AIcited by 38

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of sparse training techniques and demonstrate that gradual magnitude pruning enables value-based agents to maximize parameter effectiveness. This results in networks that yield dramatic performance improvements over traditional networks, using only a small fraction of the full network parameters.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 61 of its 87 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

61 of 87 references.

References · 61

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Layer Normalizationcited by 5,579
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  10. 2023Mastering Diverse Domains through World Modelscited by 1,193
  11. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  12. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
38 more
  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  5. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2019Model-Based Reinforcement Learning for Ataricited by 495
  8. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  9. 2019The State of Sparsity in Deep Neural Networkscited by 471
  10. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  11. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 325
  12. 2017Noisy Networks for Explorationcited by 262
  13. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  14. 2020Revisiting Fundamentals of Experience Replaycited by 171
  15. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  16. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
  17. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  18. 2023Understanding plasticity in neural networkscited by 170
  19. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  20. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  21. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  22. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  23. 2019When to use parametric models in reinforcement learning?cited by 118
  24. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  25. 2019Observational Overfitting in Reinforcement Learningcited by 90
  26. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  27. 2018Kickstarting Deep Reinforcement Learningcited by 75
  28. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  29. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  30. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 72
  31. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  32. 2020Munchausen Reinforcement Learningcited by 68
  33. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  34. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  35. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  36. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
  37. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  38. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 18; 2026: 9.

38 papers.

Sort:

By area:

  1. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  2. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  3. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  4. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 22
  5. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  6. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  7. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 17
  8. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  9. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  10. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 12
  11. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  12. 2024Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularizationcited by 11
26 more
  1. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
  2. 2024Active Fine-Tuning of Multi-Task Policiescited by 8
  3. 2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learningcited by 6
  4. 2025Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learningcited by 6
  5. 2025Hadamax Encoding: Elevating Performance in Model-Free Ataricited by 6
  6. 2025The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learningcited by 6
  7. 2024CALE: Continuous Arcade Learning Environmentcited by 5
  8. 2025Spectral Collapse Drives Loss of Plasticity in Deep Continual Learningcited by 5
  9. 2026Stable Deep Reinforcement Learning via Isotropic Gaussian Representationscited by 5
  10. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streamingcited by 4
  11. 2025Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learningcited by 4
  12. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  13. 2026The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learningcited by 2
  14. 2024Hadamard Representation: Scaffolding Performance Across Model-free RLcited by 1
  15. 2025Multi-Agent Actor-Critic with Harmonic Annealing Pruning for Dynamic Spectrum Access Systemscited by 1
  16. 2025Geometric Analysis of Neural Regression Collapse via Intrinsic Dimensioncited by 1
  17. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  18. 2024Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learningcited by 0
  19. 2025Online Training and Pruning of Deep Reinforcement Learning Networkscited by 0
  20. 2026Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learningcited by 0
  21. 2026Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitionscited by 0
  22. 2026Representation Learning Enables Scalable Multitask Deep Reinforcement Learningcited by 0
  23. 2026Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantizationcited by 0
  24. 2026Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learningcited by 0
  25. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0
  26. 2026Sustaining Plasticity via Learnable Wavelet Activations in Continual Learningcited by 0