# Measuring Visual Generalization in Continuous Control from Pixels

arXiv: https://arxiv.org/abs/2010.06740 · HTML: https://mlsanity.com/paper/2010.06740/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 21 of 21)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [Decoupling Value and Policy for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2102.10330/) (2021)
- [Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning](https://mlsanity.com/paper/2305.18499/) (2023)
- [AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents](https://mlsanity.com/paper/2310.09971/) (2023)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning](https://mlsanity.com/paper/2209.09203/) (2022)
- [On the Importance of Exploration for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2306.05483/) (2023)
- [The Generalization Gap in Offline Reinforcement Learning](https://mlsanity.com/paper/2312.05742/) (2023)
- [A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning](https://mlsanity.com/paper/2210.04561/) (2022)
- [Learning Representations for Pixel-based Control: What Matters and Why?](https://mlsanity.com/paper/2111.07775/) (2021)
- [HarmonyDream: Task Harmonization Inside World Models](https://mlsanity.com/paper/2310.00344/) (2023)
- [Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter](https://mlsanity.com/paper/2208.03374/) (2022)
- [VIBR: Learning View-Invariant Value Functions for Robust Visual Control](https://mlsanity.com/paper/2306.08537/) (2023)
- [Representation Convergence: Mutual Distillation is Secretly a Form of Regularization](https://mlsanity.com/paper/2501.02481/) (2025)
- [State Combinatorial Generalization In Decision Making With Conditional Diffusion Models](https://mlsanity.com/paper/2501.13241/) (2025)
- [A Dual-Agent Adversarial Framework for Robust Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/2501.17384/) (2025)
- [ED2: Environment Dynamics Decomposition World Models for Continuous Control](https://mlsanity.com/paper/2112.02817/) (2021)
- [Geometry of Uncertainty: Learning Metric Spaces for Multimodal State Estimation in RL](https://mlsanity.com/paper/2602.12087/) (2026)
- [Local Motion Matters: A Deconstruct-Recompose Paradigm for Reinforcement Learning Pre-training from Videos](https://mlsanity.com/paper/2607.00808/) (2026)
- [From Pixels to Temporal Correlations: Learning Informative Representations for Reinforcement Learning Pre-training](https://mlsanity.com/paper/2607.00811/) (2026)

## Abstract (arXiv)

Self-supervised learning and data augmentation have significantly reduced the performance gap between state and image-based reinforcement learning agents in continuous control tasks. However, it is still unclear whether current techniques can face a variety of visual conditions required by real-world environments. We propose a challenging benchmark that tests agents' visual generalization by adding graphical variety to existing continuous control domains. Our empirical analysis shows that current methods struggle to generalize across a diverse set of visual changes, and we examine the specific factors of variation that make these tasks difficult. We find that data augmentation techniques outperform self-supervised learning approaches and that more significant image transformations provide better visual generalization footnoteThe benchmark and our augmented actor-critic implementation are open-sourced @ https://github.com/QData/dmcremastered)