mlsanityData to 7 Oct 2026

Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning

arXiv 2107.09645 ↗First posted 20 Jul 2021cs.AI cs.LGcited by 359

Not on the map. Why?

Cited by 58 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control. DrQ-v2 builds on DrQ, an off-policy actor-critic approach that uses data augmentation to learn directly from pixels. We introduce several improvements that yield state-of-the-art results on the DeepMind Control Suite. Notably, DrQ-v2 is able to solve complex humanoid locomotion tasks directly from pixel observations, previously unattained by model-free RL. DrQ-v2 is conceptually simple, easy to implement, and provides significantly better computational footprint compared to prior work, with the majority of tasks taking just 8 hours to train on a single GPU. Finally, we publicly release DrQ-v2's implementation to provide RL practitioners with a strong and computationally efficient baseline.

Key facts. Cited by 359 papers in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

33 of 47 references.

References · 33

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  10. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2018DeepMind Control Suitecited by 935
21 more
  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2020Reinforcement Learning with Augmented Datacited by 452
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  6. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  7. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  8. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  9. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  10. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  11. 2021Reinforcement Learning with Prototypical Representationscited by 179
  12. 2020Revisiting Fundamentals of Experience Replaycited by 171
  13. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  14. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  15. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  16. 2020Visual Imitation Made Easycited by 111
  17. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  18. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  19. 2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 46
  20. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  21. 2020Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigationcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 53; 2023: 82; 2024: 91; 2025: 72; 2026: 49.

359 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  3. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  4. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  5. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  6. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  7. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  8. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  9. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  10. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 184
  11. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  12. 2022Masked World Models for Visual Controlcited by 169
38 more
  1. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  2. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  3. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  4. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  5. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  6. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  7. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  8. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  9. 2021Discovering and Achieving Goals via World Modelscited by 131
  10. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  11. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  12. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  13. 2023Synthetic Experience Replaycited by 100
  14. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
  15. 2022Deep Hierarchical Planning from Pixelscited by 96
  16. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
  17. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  18. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  19. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  20. 2023Contrastive Preference Learning: Learning from Human Feedback without RLcited by 77
  21. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  22. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  23. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  24. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  25. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  26. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  27. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  28. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
  29. 2023Elastic Decision Transformercited by 66
  30. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  31. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  32. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  33. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  34. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  35. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  36. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  37. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  38. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55

50 most cited of 359 All 359 (JSON)