# PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2001.05012 · HTML: https://mlsanity.com/paper/2001.05012/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2020.

## Built on by (most cited first, 30 of 30)

- [The State of Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2206.10369/) (2022)
- [Dynamic Sparse Training for Deep Reinforcement Learning](https://mlsanity.com/paper/2106.04217/) (2021)
- [Zero Time Waste: Recycling Predictions in Early Exit Neural Networks](https://mlsanity.com/paper/2106.05409/) (2021)
- [In value-based deep reinforcement learning, a pruned network is a good network](https://mlsanity.com/paper/2402.12479/) (2024)
- [RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch](https://mlsanity.com/paper/2205.15043/) (2022)
- [Single-Shot Pruning for Offline Reinforcement Learning](https://mlsanity.com/paper/2112.15579/) (2021)
- [Neuroplastic Expansion in Deep Reinforcement Learning](https://mlsanity.com/paper/2410.07994/) (2024)
- [Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2303.00912/) (2023)
- [Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks](https://mlsanity.com/paper/2110.14541/) (2021)
- [GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning](https://mlsanity.com/paper/2101.09650/) (2021)
- [The Impact of Quantization and Pruning on Deep Reinforcement Learning Models](https://mlsanity.com/paper/2407.04803/) (2024)
- [Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.08540/) (2024)
- [Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning](https://mlsanity.com/paper/2503.01437/) (2025)
- [Rate-Constrained Remote Contextual Bandits](https://mlsanity.com/paper/2204.12620/) (2022)
- [Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing](https://mlsanity.com/paper/2309.08477/) (2023)
- [Tiny Multi-Agent DRL for Twins Migration in UAV Metaverses: A Multi-Leader Multi-Follower Stackelberg Game Approach](https://mlsanity.com/paper/2401.09680/) (2024)
- [Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing](https://mlsanity.com/paper/2206.12146/) (2022)
- [Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving](https://mlsanity.com/paper/2402.05146/) (2024)
- [Sparse Training for Federated Learning with Regularized Error Correction](https://mlsanity.com/paper/2312.13795/) (2023)
- [Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2508.06871/) (2025)
- [Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformers](https://mlsanity.com/paper/2402.13201/) (2024)
- [Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training](https://mlsanity.com/paper/2409.19391/) (2024)
- [Multi-Agent Actor-Critic with Harmonic Annealing Pruning for Dynamic Spectrum Access Systems](https://mlsanity.com/paper/2503.15172/) (2025)
- [A Masked Pruning Approach for Dimensionality Reduction in Communication-Efficient Federated Learning Systems](https://mlsanity.com/paper/2312.03889/) (2023)
- [Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments](https://mlsanity.com/paper/2410.02516/) (2024)
- [Bi-LSTM based Multi-Agent DRL with Computation-aware Pruning for Agent Twins Migration in Vehicular Embodied AI Networks](https://mlsanity.com/paper/2505.06378/) (2025)
- [Searching for a Hidden Markov Anomaly over Multiple Processes](https://mlsanity.com/paper/2506.17108/) (2025)
- [Control-Optimized Deep Reinforcement Learning for Artificially Intelligent Autonomous Systems](https://mlsanity.com/paper/2507.00268/) (2025)
- [RNM-TD3: N:M Semi-structured Sparse Reinforcement Learning From Scratch](https://mlsanity.com/paper/2602.14578/) (2026)
- [A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies](https://mlsanity.com/paper/2609.00718/) (2026)

## Abstract (arXiv)

The recent success of deep neural networks (DNNs) for function approximation in reinforcement learning has triggered the development of Deep Reinforcement Learning (DRL) algorithms in various fields, such as robotics, computer games, natural language processing, computer vision, sensing systems, and wireless networking. Unfortunately, DNNs suffer from high computational cost and memory consumption, which limits the use of DRL algorithms in systems with limited hardware resources. In recent years, pruning algorithms have demonstrated considerable success in reducing the redundancy of DNNs in classification tasks. However, existing algorithms suffer from a significant performance reduction in the DRL domain. In this paper, we develop the first effective solution to the performance reduction problem of pruning in the DRL domain, and establish a working algorithm, named Policy Pruning and Shrinking (PoPS), to train DRL models with strong performance while achieving a compact representation of the DNN. The framework is based on a novel iterative policy pruning and shrinking method that leverages the power of transfer learning when training the DRL model. We present an extensive experimental study that demonstrates the strong performance of PoPS using the popular Cartpole, Lunar Lander, Pong, and Pacman environments. Finally, we develop an open source software for the benefit of researchers and developers in related fields.