mlsanityData to 14 Oct 2026

rQdia: Regularizing Q-Value Distributions With Image Augmentation

arXiv 2506.21367 ↗First posted 26 Jun 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

rQdia regularizes Q-value distributions with augmented images in pixel-based deep reinforcement learning. With a simple auxiliary loss, that equalizes these distributions via MSE, rQdia boosts DrQ and SAC on 9/12 and 10/12 tasks respectively in the MuJoCo Continuous Control Suite from pixels, and Data-Efficient Rainbow on 18/26 Atari Arcade environments. Gains are measured in both sample efficiency and longer-term training. Moreover, the addition of rQdia finally propels model-free continuous control from pixels over the state encoding baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 39 references that are in the map. First posted on arXiv 26 Jun 2025.

Builds on

29 of 39 references.

References · 29

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2015Prioritized Experience Replaycited by 1,099
  10. 2018DeepMind Control Suitecited by 956
  11. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
17 more
  1. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  3. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  5. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  6. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  7. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  8. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  9. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  10. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 182
  11. 2017Parameter Space Noise for Explorationcited by 161
  12. 2019When to use parametric models in reinforcement learning?cited by 118
  13. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  14. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  15. 2018Learning Actionable Representations from Visual Observationscited by 35
  16. 2021Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmarkcited by 23
  17. 2019Robust Visual Domain Randomization for Reinforcement Learningcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning
  2. 2021Q-Value Weighted Regression: Reinforcement Learning with Limited Data
  3. 2024Revisiting Data Augmentation in Deep Reinforcement Learning
  4. 2026Periodic Regularized Q-Learning
  5. 2025Residual Policy Gradient: A Reward View of KL-regularized Objective
  6. 2024Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning
  7. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
  8. 2026A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning