# QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.01055 · HTML: https://mlsanity.com/paper/1910.01055/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 19 of 19)

- [Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control](https://mlsanity.com/paper/2412.01034/) (2024)
- [Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control](https://mlsanity.com/paper/2505.15304/) (2025)
- [The Impact of Quantization and Pruning on Deep Reinforcement Learning Models](https://mlsanity.com/paper/2407.04803/) (2024)
- [Efficient Speech Representation Learning with Low-Bit Quantization](https://mlsanity.com/paper/2301.00652/) (2022)
- [LANPO: Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMs](https://mlsanity.com/paper/2510.16552/) (2025)
- [POLARON: Precision-aware On-device Learning and Adaptive Runtime-cONfigurable AI acceleration](https://mlsanity.com/paper/2506.08785/) (2025)
- [Neural Network Compression for Reinforcement Learning Tasks](https://mlsanity.com/paper/2405.07748/) (2024)
- [ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents](https://mlsanity.com/paper/2605.24900/) (2026)
- [SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems](https://mlsanity.com/paper/2405.03967/) (2024)
- [QForce-RL: Quantized FPGA-Optimized Reinforcement Learning Compute Engine](https://mlsanity.com/paper/2506.07046/) (2025)
- [The Impact of Quantization on Large Reasoning Model Reinforcement Learning](https://mlsanity.com/paper/2511.15694/) (2025)
- [Broad Critic Deep Actor Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/2411.15806/) (2024)
- [HEPPO-GAE: Hardware-Efficient Proximal Policy Optimization with Generalized Advantage Estimation](https://mlsanity.com/paper/2501.12703/) (2025)
- [Open and Sustainable AI: challenges, opportunities and the road ahead in the life sciences (October 2025 -- Version 2)](https://mlsanity.com/paper/2505.16619/) (2025)
- [Learning Quantized Continuous Controllers for Integer Hardware](https://mlsanity.com/paper/2511.07046/) (2025)
- [Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control](https://mlsanity.com/paper/2512.01467/) (2025)
- [ProSky: NEAT Meets NOMA-mmWave in the Sky of 6G](https://mlsanity.com/paper/2210.11406/) (2022)
- [AcceRL: Policy Acceleration Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2211.15023/) (2022)
- [AP-DRL: A Synergistic Algorithm-Hardware Framework for Automatic Task Partitioning of Deep Reinforcement Learning on Versal ACAP](https://mlsanity.com/paper/2603.29369/) (2026)

## Abstract (arXiv)

Deep reinforcement learning continues to show tremendous potential in achieving task-level autonomy, however, its computational and energy demands remain prohibitively high. In this paper, we tackle this problem by applying quantization to reinforcement learning. To that end, we introduce a novel Reinforcement Learning (RL) training paradigm, textitActorQ, to speed up actor-learner distributed RL training. textitActorQ leverages 8-bit quantized actors to speed up data collection without affecting learning convergence. Our quantized distributed RL training system, textitActorQ, demonstrates end-to-end speedups bluebetween 1.5 × and 5.41×, and faster convergence over full precision training on a range of tasks (Deepmind Control Suite) and different RL algorithms (D4PG, DQN). Furthermore, we compare the carbon emissions (Kgs of CO2) of textitActorQ versus standard reinforcement learning bluealgorithms on various tasks. Across various settings, we show that textitActorQ enables more environmentally friendly reinforcement learning by achieving bluecarbon emission improvements between 1.9× and 3.76× compared to training RL-agents in full-precision. We believe that this is the first of many future works on enabling computationally energy-efficient and sustainable reinforcement learning. The source code is available here for the public to use: urlhttps://github.com/harvard-edge/QuaRL.