mlsanityData to 7 Oct 2026

On the Interplay Between Sparsity and Training in Deep Reinforcement Learning

arXiv 2501.16729 ↗First posted 28 Jan 2025cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

We study the benefits of different sparse architectures for deep reinforcement learning. In particular, we focus on image-based domains where spatially-biased and fully-connected architectures are common. Using these and several other architectures of equal capacity, we show that sparse structure has a significant effect on learning performance. We also observe that choosing the best sparse architecture for a given domain depends on whether the hidden layer weights are fixed or learned.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

21 of 37 references.

References · 21

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Going Deeper with Convolutionscited by 4,231
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  5. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  6. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  7. 2019Weight Agnostic Neural Networkscited by 71
  8. 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 66
  9. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  10. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  11. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  12. 2022Dynamic Sparse Network for Time Series Classification: Learning What to "see''cited by 22
9 more
  1. 2021Neural Network Training Using ℓ1-Regularization and Bi-fidelity Datacited by 16
  2. 2022The Quest for a Common Model of the Intelligent Decision Makercited by 16
  3. 2023Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learningcited by 13
  4. 2021Faster Deep Reinforcement Learning with Slower Online Networkcited by 7
  5. 2023On the Convergence of Bounded Agentscited by 7
  6. 2019Learning Sparse Representations Incrementally in Deep Reinforcement Learningcited by 3
  7. 2023Towards model-free RL algorithms that scale well with unstructured datacited by 3
  8. 2021Adapting the Function Approximation Architecture in Online Reinforcement Learningcited by 2
  9. 2024Are Sparse Neural Networks Better Hard Sample Learners?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 46
  2. 2025Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Modelscited by 2
  3. 2026Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learningcited by 0