mlsanityData to 7 Oct 2026

A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning

arXiv 2210.04561 ↗First posted 10 Oct 2022cs.CV cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

Visual reinforcement learning (RL), which makes decisions directly from high-dimensional visual inputs, has demonstrated significant potential in various domains. However, deploying visual RL techniques in the real world remains challenging due to their low sample efficiency and large generalization gaps. To tackle these obstacles, data augmentation (DA) has become a widely used technique in visual RL for acquiring sample-efficient and generalizable policies by diversifying the training data. This survey aims to provide a timely and essential review of DA techniques in visual RL in recognition of the thriving development in this field. In particular, we propose a unified framework for analyzing visual RL and understanding the role of DA in it. We then present a principled taxonomy of the existing augmentation techniques used in visual RL and conduct an in-depth discussion on how to better leverage augmented data in different scenarios. Moreover, we report a systematic empirical evaluation of DA-based techniques in visual RL and conclude by highlighting the directions for future research. As the first comprehensive survey of DA in visual RL, this work is expected to offer valuable guidance to this emerging field.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 131 of its 202 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

131 of 202 references.

References · 131

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  9. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  10. 2015Continuous control with deep reinforcement learningcited by 3,279
  11. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
38 more
  1. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2017Random Erasing Data Augmentationcited by 1,235
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  6. 2020Mastering Atari with Discrete World Modelscited by 990
  7. 2018DeepMind Control Suitecited by 935
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  9. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  13. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  14. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  15. 2021Domain Generalization with MixStylecited by 563
  16. 2019Model-Based Reinforcement Learning for Ataricited by 495
  17. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  18. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  19. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  20. 2020Reinforcement Learning with Augmented Datacited by 452
  21. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  22. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  23. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  24. 2016Learning to reinforcement learncited by 369
  25. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  26. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  27. 2019On Mutual Information Maximization for Representation Learningcited by 315
  28. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  29. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  30. 2017Data Augmentation Generative Adversarial Networkscited by 278
  31. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  32. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  33. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  34. 2022Masked Visual Pre-training for Motor Controlcited by 246
  35. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  36. 2015Contextual Markov Decision Processescited by 213
  37. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  38. 2018An Introduction to Deep Reinforcement Learningcited by 201

50 most cited of 131 All 131 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 3; 2025: 8; 2026: 5.

23 papers.

Sort:

By area:

  1. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  2. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  3. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  4. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 36
  5. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  6. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 20
  7. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  8. 2023Normalization Enhances Generalization in Visual Reinforcement Learningcited by 13
  9. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  10. 2023ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learningcited by 9
  11. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8
  12. 2023MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learningcited by 6
11 more
  1. 2024Revisiting Data Augmentation in Deep Reinforcement Learningcited by 5
  2. 2025ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representationscited by 5
  3. 2026SA-VLA: Spatially-Aware Flow-Matching for Vision-Language-Action Reinforcement Learningcited by 3
  4. 2026Data Augmentation: A Fourier Analysis Perspectivecited by 3
  5. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Roboticscited by 2
  6. 2025Representation Convergence: Mutual Distillation is Secretly a Form of Regularizationcited by 0
  7. 2025Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learningcited by 0
  8. 2025Taming Data Challenges in ML-based Security Tasks Using Generative AIcited by 0
  9. 2025Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulationcited by 0
  10. 2025Synthetic Data is Sufficient for Zero-Shot Visual Generalization from Offline Datacited by 0
  11. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0