mlsanityData to 7 Oct 2026

Generalization in Reinforcement Learning by Soft Data Augmentation

arXiv 2011.13389 ↗First posted 26 Nov 2020cs.LGcited by 132

Not on the map. Why?

Abstract (arXiv)

Extensive efforts have been made to improve the generalization ability of Reinforcement Learning (RL) methods via domain randomization and data augmentation. However, as more factors of variation are introduced during training, optimization becomes increasingly challenging, and empirically may result in lower sample efficiency and unstable training. Instead of learning policies directly from augmented data, we propose SOft Data Augmentation (SODA), a method that decouples augmentation from policy learning. Specifically, SODA imposes a soft constraint on the encoder that aims to maximize the mutual information between latent representations of augmented and non-augmented data, while the RL optimization process uses strictly non-augmented data. Empirical evaluations are performed on diverse tasks from DeepMind Control suite as well as a robotic manipulation task, and we find SODA to significantly advance sample efficiency, generalization, and stability in training over state-of-the-art vision-based RL methods.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 27; 2023: 33; 2024: 27; 2025: 22; 2026: 12.

132 papers.

Sort:

By area:

  1. 2021Domain Generalization: A Surveycited by 760
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  6. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  7. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  8. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  9. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  10. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  11. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  12. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
38 more
  1. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  2. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  3. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  4. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  5. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  6. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  7. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
  8. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  9. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  10. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  11. 2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
  12. 2022Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretrainingcited by 37
  13. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  14. 2023EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulationcited by 37
  15. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 36
  16. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  17. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  18. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  19. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  20. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  21. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  22. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  23. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
  24. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
  25. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  26. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  27. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  28. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  29. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  30. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 20
  31. 2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
  32. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 18
  33. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 18
  34. 2025Do You Need Proprioceptive States in Visuomotor Policies?cited by 16
  35. 2023MoVie: Visual Model-Based Policy Adaptation for View Generalizationcited by 15
  36. 2022Learning to Generalize with Object-centric Agents in the Open World Survival Game Craftercited by 13
  37. 2022Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racingcited by 13
  38. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13

50 most cited of 132 All 132 (JSON)