mlsanityData to 7 Oct 2026

The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels

arXiv 2101.02722 ↗First posted 7 Jan 2021cs.RO cs.AI cs.CVcited by 97

Not on the map. Why?

Abstract (arXiv)

Robots have to face challenging perceptual settings, including changes in viewpoint, lighting, and background. Current simulated reinforcement learning (RL) benchmarks such as DM Control provide visual input without such complexity, which limits the transfer of well-performing methods to the real world. In this paper, we extend DM Control with three kinds of visual distractions (variations in background, color, and camera pose) to produce a new challenging benchmark for vision-based control, and we analyze state of the art RL algorithms in these settings. Our experiments show that current RL methods for vision-based control perform poorly under distractions, and that their performance decreases with increasing distraction complexity, showing that new methods are needed to cope with the visual complexities of the real world. We also find that combinations of multiple distraction types are more difficult than a mere combination of their individual effects.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 7 Jan 2021.

Builds on

16 of 23 references.

References · 16

  1. 2016Layer Normalizationcited by 5,579
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2018DeepMind Control Suitecited by 935
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2020Reinforcement Learning with Augmented Datacited by 452
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
4 more
  1. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  2. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  3. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  4. 2018Natural Environment Benchmarks for Reinforcement Learningcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 12; 2023: 20; 2024: 21; 2025: 10; 2026: 18.

97 papers.

Sort:

By area:

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  2. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  3. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
  4. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  5. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  6. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  7. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  8. 2021Learning Task Informed Abstractionscited by 58
  9. 2024Benchmarking Mobile Device Control Agents across Diverse Configurationscited by 54
  10. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  11. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  12. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
38 more
  1. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 31
  2. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
  3. 2021Which priors matter? Benchmarking models for learning latent dynamicscited by 26
  4. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
  5. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
  6. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  7. 2021Contrastive Active Inferencecited by 21
  8. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 20
  9. 2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
  10. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 18
  11. 2021Improving Zero-shot Generalization in Offline Reinforcement Learning using Generalized Similarity Functionscited by 17
  12. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  13. 2023FOCUS: Object-Centric World Models for Robotics Manipulationcited by 17
  14. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  15. 2023Curious Replay for Model-based Adaptationcited by 16
  16. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  17. 2023MoVie: Visual Model-Based Policy Adaptation for View Generalizationcited by 15
  18. 2025Horizon Generalization in Reinforcement Learningcited by 15
  19. 2021Imitation by Predicting Observationscited by 13
  20. 2022Learning to Generalize with Object-centric Agents in the Open World Survival Game Craftercited by 13
  21. 2023Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learningcited by 13
  22. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  23. 2024When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learningcited by 13
  24. 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 13
  25. 2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
  26. 2024A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learningcited by 12
  27. 2023Efficient RL via Disentangled Environment and Agent Representationscited by 11
  28. 2022Task-Induced Representation Learningcited by 10
  29. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  30. 2024Simplifying Latent Dynamics with Softly State-Invariant World Modelscited by 10
  31. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  32. 2023MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learningcited by 9
  33. 2025Object-Centric Latent Action Learningcited by 9
  34. 2021Invariance Through Latent Alignmentcited by 8
  35. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  36. 2022Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learningcited by 7
  37. 2025Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learningcited by 7
  38. 2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6

50 most cited of 97 All 97 (JSON)