mlsanityData to 14 Oct 2026

RNM-TD3: N:M Semi-structured Sparse Reinforcement Learning From Scratch

arXiv 2602.14578 ↗First posted 16 Feb 2026cs.LG cs.ARcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Sparsity is a well-studied technique for compressing deep neural networks (DNNs) without compromising performance. In deep reinforcement learning (DRL), neural networks with up to 5% of their original weights can still be trained with minimal performance loss compared to their dense counterparts. However, most existing methods rely on unstructured fine-grained sparsity, which limits hardware acceleration opportunities due to irregular computation patterns. Structured coarse-grained sparsity enables hardware acceleration, yet typically degrades performance and increases pruning complexity. In this work, we present, to the best of our knowledge, the first study on N:M structured sparsity in RL, which balances compression, performance, and hardware efficiency. Our framework enforces row-wise N:M sparsity throughout training for all networks in off-policy RL (TD3), maintaining compatibility with accelerators that support N:M sparse matrix operations. Experiments on continuous-control benchmarks show that RNM-TD3, our N:M sparse agent, outperforms its dense counterpart at 50%-75% sparsity (e.g., 2:4 and 1:4), achieving up to a 14% increase in performance at 2:4 sparsity on the Ant environment. RNM-TD3 remains competitive even at 87.5% sparsity (1:8), while enabling potential training speedups.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 39 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

32 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning20 Jun

Other references · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  8. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  9. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  10. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  11. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  12. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
19 more
  1. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  2. 2019The State of Sparsity in Deep Neural Networkscited by 476
  3. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  4. 2015Policy Distillationcited by 440
  5. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  6. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  7. 2023DepGraph: Towards Any Structural Pruningcited by 203
  8. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
  9. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  10. 2018Kickstarting Deep Reinforcement Learningcited by 81
  11. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  12. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 61
  13. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  14. 2020PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learningcited by 30
  15. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  16. 2021On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learningcited by 24
  17. 2021Single-Shot Pruning for Offline Reinforcement Learningcited by 21
  18. 2023Bi-directional Masks for Efficient N:M Sparse Trainingcited by 13
  19. 2025Systolic Sparse Tensor Slices: FPGA Building Blocks for Sparse and Dense AI Accelerationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Neural Network Compression for Reinforcement Learning Tasks
  2. 2026Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts
  3. 2025On the Interplay Between Sparsity and Training in Deep Reinforcement Learning
  4. 2024Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving
  5. 2024Optimal Policy Sparsification and Low Rank Decomposition for Deep Reinforcement Learning
  6. 2025Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
  7. 2024The Impact of Quantization and Pruning on Deep Reinforcement Learning Models
  8. 2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning