mlsanityData to 7 Oct 2026

iQRL -- Implicitly Quantized Representations for Sample-efficient Reinforcement Learning

arXiv 2406.02696 ↗First posted 4 Jun 2024cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Learning representations for reinforcement learning (RL) has shown much promise for continuous control. We propose an efficient representation learning method using only a self-supervised latent-state consistency loss. Our approach employs an encoder and a dynamics model to map observations to latent states and predict future latent states, respectively. We achieve high performance and prevent representation collapse by quantizing the latent representation such that the rank of the representation is empirically preserved. Our method, named iQRL: implicitly Quantized Reinforcement Learning, is straightforward, compatible with any model-free RL algorithm, and demonstrates excellent performance by outperforming other recently proposed representation learning methods in continuous control benchmarks from DeepMind Control Suite.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

38 of 51 references.

References · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2023Mastering Diverse Domains through World Modelscited by 1,193
  6. 2020Mastering Atari with Discrete World Modelscited by 990
  7. 2018DeepMind Control Suitecited by 935
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  12. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
26 more
  1. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  3. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  4. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  5. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  6. 2018Towards a Definition of Disentangled Representationscited by 294
  7. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  8. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  9. 2021Mastering Atari Games with Limited Datacited by 235
  10. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  11. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
  12. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  13. 2019Unsupervised State Representation Learning in Ataricited by 144
  14. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  15. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  16. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 67
  17. 2017Near Optimal Behavior via Approximate State Abstractioncited by 63
  18. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  19. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  20. 2022The Mechanism of Prediction Head in Non-contrastive Self-supervised Learningcited by 38
  21. 2018Natural Environment Benchmarks for Reinforcement Learningcited by 32
  22. 2020Representations for Stable Off-Policy Reinforcement Learningcited by 31
  23. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
  24. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  25. 2021Learning Dynamics Models for Model Predictive Agentscited by 21
  26. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

  1. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  2. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  3. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 35
  4. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  5. 2025Discrete Codebook World Models for Continuous Controlcited by 9
  6. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  7. 2026Debiased Model-based Representations for Sample-efficient Continuous Controlcited by 1
  8. 2026Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learningcited by 0
  9. 2026RAY-TOLD: Ray-Based Latent Dynamics for Dense Dynamic Obstacle Avoidance with TDMPCcited by 0