# Robust Visual Domain Randomization for Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.10537 · HTML: https://mlsanity.com/paper/1910.10537/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 13 of 13)

- [ODRL: A Benchmark for Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2410.20750/) (2024)
- [Domain Adversarial Reinforcement Learning](https://mlsanity.com/paper/2102.07097/) (2021)
- [Offline Reinforcement Learning with Domain-Unlabeled Data](https://mlsanity.com/paper/2404.07465/) (2024)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System](https://mlsanity.com/paper/2312.10472/) (2023)
- [Representation Convergence: Mutual Distillation is Secretly a Form of Regularization](https://mlsanity.com/paper/2501.02481/) (2025)
- [A Dual-Agent Adversarial Framework for Robust Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/2501.17384/) (2025)
- [Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Survey](https://mlsanity.com/paper/2602.00399/) (2026)
- [DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning](https://mlsanity.com/paper/2410.17186/) (2024)
- [rQdia: Regularizing Q-Value Distributions With Image Augmentation](https://mlsanity.com/paper/2506.21367/) (2025)
- [A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning](https://mlsanity.com/paper/2606.16933/) (2026)
- [ENVS: Environment-Native Verified Search for Long-Horizon GUI Agents](https://mlsanity.com/paper/2606.22948/) (2026)
- [DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning](https://mlsanity.com/paper/2607.16090/) (2026)

## Abstract (arXiv)

Producing agents that can generalize to a wide range of visually different environments is a significant challenge in reinforcement learning. One method for overcoming this issue is visual domain randomization, whereby at the start of each training episode some visual aspects of the environment are randomized so that the agent is exposed to many possible variations. However, domain randomization is highly inefficient and may lead to policies with high variance across domains. Instead, we propose a regularization method whereby the agent is only trained on one variation of the environment, and its learned state representations are regularized during training to be invariant across domains. We conduct experiments that demonstrate that our technique leads to more efficient and robust learning than standard domain randomization, while achieving equal generalization scores.