mlsanityData to 13 Oct 2026

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

arXiv 2210.08065 ↗First posted 14 Oct 2022cs.RO cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) is one of the most powerful tools for synthesizing complex robotic behaviors. But training DRL models is incredibly compute and memory intensive, requiring large training datasets and replay buffers to achieve performant results. This poses a challenge for the next generation of field robots that will need to learn on the edge to adapt to their environment. In this paper, we begin to address this issue through observation space quantization. We evaluate our approach using four simulated robot locomotion tasks and two state-of-the-art DRL algorithms, the on-policy Proximal Policy Optimization (PPO) and off-policy Soft Actor-Critic (SAC) and find that observation space quantization reduces overall memory costs by as much as 4.2x without impacting learning performance.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

35 of 45 references.

References · 35

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2015Continuous control with deep reinforcement learningcited by 3,290
  3. 2015Trust Region Policy Optimizationcited by 2,513
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,310
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,223
  6. 2016OpenAI Gymcited by 1,694
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,585
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,241
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,185
  10. 2019Solving Rubik's Cube with a Robot Handcited by 702
  11. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 660
  12. 2018Learning Dexterous In-Hand Manipulationcited by 645
23 more
  1. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 422
  2. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 399
  3. 2022DayDreamer: World Models for Physical Robot Learningcited by 375
  4. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 301
  5. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 290
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 279
  7. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 270
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 193
  10. 2020Revisiting Fundamentals of Experience Replaycited by 173
  11. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  12. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 80
  13. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 77
  14. 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 51
  15. 2021LASER: Learning a Latent Action Space for Efficient Reinforcement Learningcited by 32
  16. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 27
  17. 2022Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robotscited by 17
  18. 2019A Comparison of Action Spaces for Learning Manipulation Taskscited by 16
  19. 2021Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimizationcited by 16
  20. 2021Learning Visually Guided Latent Actions for Assistive Teleoperationcited by 12
  21. 2021OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulationcited by 7
  22. 2017FPGA Architecture for Deep Learning and its application to Planetary Roboticscited by 3
  23. 2021Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precisioncited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 1.

3 papers.

  1. 2023Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagerycited by 3
  2. 2023Differentially Encoded Observation Spaces for Perceptive Reinforcement Learningcited by 1
  3. 2025Learning Quantized Continuous Controllers for Integer Hardwarecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies
  2. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  3. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
  4. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning
  5. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
  6. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
  7. 2026Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control
  8. 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning