mlsanityData to 7 Oct 2026

GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning

arXiv 2101.09650 ↗First posted 24 Jan 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has shown remarkable success in sequential decision-making problems but suffers from a long training time to obtain such good performance. Many parallel and distributed DRL training approaches have been proposed to solve this problem, but it is difficult to utilize them on resource-limited devices. In order to accelerate DRL in real-world edge devices, memory bandwidth bottlenecks due to large weight transactions have to be resolved. However, previous iterative pruning not only shows a low compression ratio at the beginning of training but also makes DRL training unstable. To overcome these shortcomings, we propose a novel weight compression method for DRL training acceleration, named group-sparse training (GST). GST selectively utilizes block-circulant compression to maintain a high weight compression ratio during all iterations of DRL training and dynamically adapt target sparsity through reward-aware pruning for stable training. Thanks to the features, GST achieves a 25 %p ~ 41.5 %p higher average compression ratio than the iterative pruning method without reward drop in Mujoco Halfcheetah-v2 and Mujoco humanoid-v2 environment with TD3 training.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 24 Jan 2021.

Builds on

23 of 33 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  9. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  10. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  11. 2019Challenges of Real-World Reinforcement Learningcited by 282
  12. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
11 more
  1. 2018Distributed Prioritized Experience Replaycited by 260
  2. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  3. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  4. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  5. 2017CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matricescited by 53
  6. 2016Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPUcited by 46
  7. 2015Structured Transforms for Small-Footprint Deep Learningcited by 41
  8. 2019PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfigurationcited by 41
  9. 2020PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learningcited by 30
  10. 2015ACDC: A Structured Efficient Linear Layercited by 19
  11. 2019CircConv: A Structured Convolution with Low Complexitycited by 8

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 0; 2024: 1; 2025: 1.

7 papers.

  1. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  2. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  3. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networkscited by 39
  4. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  5. 2022LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learningcited by 1
  6. 2024Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformerscited by 1
  7. 2025Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functionscited by 0