# Single-Shot Pruning for Offline Reinforcement Learning arXiv: https://arxiv.org/abs/2112.15579 ยท HTML: https://mlsanity.com/paper/2112.15579/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. - Builds on 25 of its 35 references that are in the map. - First posted on arXiv 31 Dec 2021. ## Placed in - Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/) ## Builds on (most cited first, 25 of 25) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015) - [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018) - [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015) - [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020) - [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016) - [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020) - [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018) - [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016) - [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021) - [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019) - [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018) - [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017) - [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019) - [Picking Winning Tickets Before Training by Preserving Gradient Flow](https://mlsanity.com/paper/2002.07376/) (2020) - [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017) - [Offline Reinforcement Learning with Fisher Divergence Critic Regularization](https://mlsanity.com/paper/2103.08050/) (2021) - [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://mlsanity.com/paper/1705.07565/) (2017) - [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018) - [Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks](https://mlsanity.com/paper/1905.10961/) (2019) - [Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers](https://mlsanity.com/paper/2005.06870/) (2020) - [DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures](https://mlsanity.com/paper/1908.09979/) (2019) - [Dynamic Sparse Training for Deep Reinforcement Learning](https://mlsanity.com/paper/2106.04217/) (2021) - [PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning](https://mlsanity.com/paper/2001.05012/) (2020) - [Pre-Defined Sparse Neural Networks with Hardware Acceleration](https://mlsanity.com/paper/1812.01164/) (2018) ## Built on by (most cited first, 21 of 21) - [The Dormant Neuron Phenomenon in Deep Reinforcement Learning](https://mlsanity.com/paper/2302.12902/) (2023) - [Mixtures of Experts Unlock Parameter Scaling for Deep RL](https://mlsanity.com/paper/2402.08609/) (2024) - [The State of Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2206.10369/) (2022) - [In value-based deep reinforcement learning, a pruned network is a good network](https://mlsanity.com/paper/2402.12479/) (2024) - [RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch](https://mlsanity.com/paper/2205.15043/) (2022) - [Mixture of Experts in a Mixture of RL settings](https://mlsanity.com/paper/2406.18420/) (2024) - [Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2302.06548/) (2023) - [Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning](https://mlsanity.com/paper/2506.17204/) (2025) - [Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning](https://mlsanity.com/paper/2503.01437/) (2025) - [Rate-Constrained Remote Contextual Bandits](https://mlsanity.com/paper/2204.12620/) (2022) - [Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learning](https://mlsanity.com/paper/2505.17749/) (2025) - [Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity](https://mlsanity.com/paper/2506.17155/) (2025) - [SalientGrads: Sparse Models for Communication Efficient and Data Aware Distributed Federated Training](https://mlsanity.com/paper/2304.07488/) (2023) - [SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learning](https://mlsanity.com/paper/2405.09037/) (2024) - [Hadamard Representation: Scaffolding Performance Across Model-free RL](https://mlsanity.com/paper/2406.09079/) (2024) - [Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning](https://mlsanity.com/paper/2510.12096/) (2025) - [One is More: Diverse Perspectives within a Single Network for Efficient DRL](https://mlsanity.com/paper/2310.14009/) (2023) - [ECToNAS: Evolutionary Cross-Topology Neural Architecture Search](https://mlsanity.com/paper/2403.05123/) (2024) - [Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments](https://mlsanity.com/paper/2410.02516/) (2024) - [Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts](https://mlsanity.com/paper/2507.07140/) (2025) - [RNM-TD3: N:M Semi-structured Sparse Reinforcement Learning From Scratch](https://mlsanity.com/paper/2602.14578/) (2026) ## Abstract (arXiv) Deep Reinforcement Learning (RL) is a powerful framework for solving complex real-world problems. Large neural networks employed in the framework are traditionally associated with better generalization capabilities, but their increased size entails the drawbacks of extensive training duration, substantial hardware resources, and longer inference times. One way to tackle this problem is to prune neural networks leaving only the necessary parameters. State-of-the-art concurrent pruning techniques for imposing sparsity perform demonstrably well in applications where data distributions are fixed. However, they have not yet been substantially explored in the context of RL. We close the gap between RL and single-shot pruning techniques and present a general pruning approach to the Offline RL. We leverage a fixed dataset to prune neural networks before the start of RL training. We then run experiments varying the network sparsity level and evaluating the validity of pruning at initialization techniques in continuous control tasks. Our results show that with 95% of the network weights pruned, Offline-RL algorithms can still retain performance in the majority of our experiments. To the best of our knowledge, no prior work utilizing pruning in RL retained performance at such high levels of sparsity. Moreover, pruning at initialization techniques can be easily integrated into any existing Offline-RL algorithms without changing the learning objective.