Data to 7 Oct 2026

Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels

arXiv 2004.13649 ↗First posted 28 Apr 2020cs.LG cs.CV eess.IVcited by 573

Not on the map. Why?

Cited by 79 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a simple data augmentation technique that can be applied to standard model-free reinforcement learning algorithms, enabling robust learning directly from pixels without the need for auxiliary losses or pre-training. The approach leverages input perturbations commonly used in computer vision tasks to regularize the value function. Existing model-free approaches, such as Soft Actor-Critic (SAC), are not able to train deep networks effectively from image pixels. However, the addition of our augmentation method dramatically improves SAC's performance, enabling it to reach state-of-the-art performance on the DeepMind control suite, surpassing model-based (Dreamer, PlaNet, and SLAC) methods and recently proposed contrastive learning (CURL). Our approach can be combined with any model-free reinforcement learning algorithm, requiring only minor modifications. An implementation can be found at https://sites.google.com/view/data-regularized-q.

Key facts. Cited by 573 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 92; 2022: 117; 2023: 127; 2024: 109; 2025: 80; 2026: 47.

573 papers.

Sort:

By area:

2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
2021Domain Generalization: A Surveycited by 760
2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
2021Domain Generalization with MixStylecited by 563
2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
2022Temporal Difference Learning for Model Predictive Controlcited by 375
2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
2023Efficient Online Reinforcement Learning with Offline Datacited by 332
2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331

38 more

2022Transformers are Sample-Efficient World Modelscited by 317
2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
2022The Primacy Bias in Deep Reinforcement Learningcited by 242
2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
2021Mastering Atari Games with Limited Datacited by 235
2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 184
2021Reinforcement Learning with Prototypical Representationscited by 179
2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
2022Masked World Models for Visual Controlcited by 169
2022Evolving Curricula with Regret-Based Environment Designcited by 166
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
2023Language-Driven Representation Learning for Roboticscited by 163
2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
2021APS: Active Pretraining with Successor Featurescited by 109
2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86

50 most cited of 573 All 573 (JSON)