mlsanityData to 7 Oct 2026

The Impact of Quantization and Pruning on Deep Reinforcement Learning Models

arXiv 2407.04803 ↗First posted 5 Jul 2024cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has achieved remarkable success across various domains, such as video games, robotics, and, recently, large language models. However, the computational costs and memory requirements of DRL models often limit their deployment in resource-constrained environments. The challenge underscores the urgent need to explore neural network compression methods to make RDL models more practical and broadly applicable. Our study investigates the impact of two prominent compression methods, quantization and pruning on DRL models. We examine how these techniques influence four performance factors: average return, memory, inference time, and battery utilization across various DRL algorithms and environments. Despite the decrease in model size, we identify that these compression techniques generally do not improve the energy efficiency of DRL models, but the model size decreases. We provide insights into the trade-offs between model compression and DRL performance, offering guidelines for deploying efficient DRL models in resource-constrained settings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

35 of 48 references.

References · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  8. 2015Trust Region Policy Optimizationcited by 2,503
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  12. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
23 more
  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2020Knowledge Distillation: A Surveycited by 1,578
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  7. 2020Mastering Atari with Discrete World Modelscited by 990
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  9. 2018Rethinking the Value of Network Pruningcited by 595
  10. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  11. 2021Post-Training Quantization for Vision Transformercited by 274
  12. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  13. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  14. 2023DepGraph: Towards Any Structural Pruningcited by 202
  15. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  16. 2021Vector Quantized Models for Planningcited by 47
  17. 2023Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Modelscited by 44
  18. 2020PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learningcited by 30
  19. 2021Continuous Control with Action Quantization from Demonstrationscited by 21
  20. 2019QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learningcited by 19
  21. 2022OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantizationcited by 19
  22. 2021FIXAR: A Fixed-Point Deep Reinforcement Learning Platform with Quantization-Aware Training and Adaptive Parallelismcited by 5
  23. 2024Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Drivingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Attentions Under the Microscope: A Comparative Study of Resource Utilization for Variants of Self-Attentioncited by 2
  2. 2025Learning Quantized Continuous Controllers for Integer Hardwarecited by 1
  3. 2025Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Controlcited by 1
  4. 2025TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantizationcited by 0
  5. 2026AP-DRL: A Synergistic Algorithm-Hardware Framework for Automatic Task Partitioning of Deep Reinforcement Learning on Versal ACAPcited by 0
  6. 2026BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deploymentcited by 0