Data to 14 Oct 2026

The State of Sparse Training in Deep Reinforcement Learning

arXiv 2206.10369 ↗First posted 17 Jun 2022cs.LG cs.AIcited by 61

Not on the map. Why?

Cited by 31 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The use of sparse neural networks has seen rapid growth in recent years, particularly in computer vision. Their appeal stems largely from the reduced number of parameters required to train and store, as well as in an increase in learning efficiency. Somewhat surprisingly, there have been very few efforts exploring their use in Deep Reinforcement Learning (DRL). In this work we perform a systematic investigation into applying a number of existing sparse training techniques on a variety of DRL agents and environments. Our results corroborate the findings from sparse training in the computer vision domain - sparse networks perform better than dense networks for the same parameter count - in the DRL domain. We provide detailed analyses on how the various components in DRL are affected by the use of sparse networks and conclude by suggesting promising avenues for improving the effectiveness of sparse training methods, as well as for advancing their use in DRL.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 55 of its 77 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

55 of 77 references.

References · 55

2015Deep Residual Learning for Image Recognitioncited by 52,096
2014Adam: A Method for Stochastic Optimizationcited by 37,240
2017Proximal Policy Optimization Algorithmscited by 15,215
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807

38 more

2018Recurrent World Models Facilitate Policy Evolutioncited by 785
2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
2018Rethinking the Value of Network Pruningcited by 599
2020What is the State of Neural Network Pruning?cited by 583
2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
2019Rigging the Lottery: Making All Tickets Winnerscited by 495
2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
2019The State of Sparsity in Deep Neural Networkscited by 476
2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 295
2019Stabilizing Transformers for Reinforcement Learningcited by 270
2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
2022The Primacy Bias in Deep Reinforcement Learningcited by 247
2018Efficient Neural Audio Synthesiscited by 219
2020The large learning rate phase of deep learning: the catapult mechanismcited by 216
2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 189
2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 182
2019What Do Compressed Deep Neural Networks Forget?cited by 171
2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 151
2017Deep Rewiring: Training very sparse deep networkscited by 139
2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
2017Bayesian Compression for Deep Learningcited by 116
2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 110
2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
2021RRL: Resnet as representation for Reinforcement Learningcited by 91
2020Gradient Flow in Sparse Neural Networks and How Lottery Tickets Wincited by 77
2019Weight Agnostic Neural Networkscited by 71
2020Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformerscited by 67
2019The Difficulty of Training Sparse Neural Networkscited by 62
2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 59
2019Discovering Neural Wiringscited by 56
2020Neuroevolution of Self-Interpretable Agentscited by 49
2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
2021Efficient Transformers in Reinforcement Learning using Actor-Learner Distillationcited by 35

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 14; 2025: 22; 2026: 14.

61 papers.

Sort:

By area:

2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 49
2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
2023Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researcherscited by 25
2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
2024Mixture of Experts in a Mixture of RL settingscited by 21
2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
2023Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Trainingcited by 17
2023Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learningcited by 14
2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14

38 more

2023Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learningcited by 13
2023JaxPruner: A concise library for sparsity researchcited by 13
2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
2024Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustnesscited by 11
2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
2025FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Modelscited by 11
2023Balanced Training for Sparse GANscited by 8
2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learningcited by 6
2023Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correctioncited by 5
2024CALE: Continuous Arcade Learning Environmentcited by 5
2025Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learningcited by 4
2025Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learningcited by 4
2024Neural Network Compression for Reinforcement Learning Taskscited by 3
2024Navigating Extremes: Dynamic Sparsity in Large Output Spacescited by 3
2025On the Interplay Between Sparsity and Training in Deep Reinforcement Learningcited by 3
2025The Formalism-Implementation Gap in Reinforcement Learning Researchcited by 3
2026GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMscited by 3
2026The Multiple Ticket Hypothesis: Random Sparse Subnetworks Suffice for RLVRcited by 3
2025Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolutioncited by 2
2025Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Modelscited by 2
2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
2025Online reinforcement learning via sparse Gaussian mixture model Q-functionscited by 2
2025Use the Online Network If You Can: Towards Fast and Stable Reinforcement Learningcited by 2
2023Learning Activation Functions for Sparse Neural Networkscited by 1
2024Evasive Active Hypothesis Testing with Deep Neuroevolution: The Single- and Multi-Agent Casescited by 1
2024Hadamard Representation: Scaffolding Performance Across Model-free RLcited by 1
2024Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Trainingcited by 1
2025Multi-Agent Actor-Critic with Harmonic Annealing Pruning for Dynamic Spectrum Access Systemscited by 1
2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
2025Learning Quantized Continuous Controllers for Integer Hardwarecited by 1
2025Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Controlcited by 1
2023One is More: Diverse Perspectives within a Single Network for Efficient DRLcited by 0
2024Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsitycited by 0
2025NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensemblingcited by 0
2025TopK Language Modelscited by 0
2025Online Training and Pruning of Deep Reinforcement Learning Networkscited by 0
2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
2026Topology-Aware Revival for Efficient Sparse Trainingcited by 0

50 most cited of 61 All 61 (JSON)