mlsanityData to 14 Oct 2026

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

arXiv 2104.14386 ↗First posted 29 Apr 2021cs.LG cs.AI cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Visual domain randomization in simulated environments is a widely used method to transfer policies trained in simulation to real robots. However, domain randomization and augmentation hamper the training of a policy. As reinforcement learning struggles with a noisy training signal, this additional nuisance can drastically impede training. For difficult tasks it can even result in complete failure to learn. To overcome this problem we propose to pre-train a perception encoder that already provides an embedding invariant to the randomization. We demonstrate that this yields consistently improved results on a randomized version of DeepMind control suite tasks and a stacking environment on arbitrary backgrounds with zero-shot transfer to a physical robot.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

38 of 43 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2017Mask R-CNNcited by 7,277
  8. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  9. 2013Auto-Encoding Variational Bayescited by 4,053
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
26 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  2. 2018DeepMind Control Suitecited by 956
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2020Reinforcement Learning with Augmented Datacited by 468
  8. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  9. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  10. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  11. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  12. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  13. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  14. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  15. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  16. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  17. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 117
  18. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  19. 2018Driving Policy Transfer via Modularity and Abstractioncited by 92
  20. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 91
  21. 2019Does computer vision matter for action?cited by 64
  22. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 52
  23. 2018VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Controlcited by 35
  24. 2017Learning a visuomotor controller for real world robotic grasping using simulated depth imagescited by 29
  25. 2019Motion Perception in Reinforcement Learning with Dynamic Objectscited by 13
  26. 2019Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Controlcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Bridging Sim2Real Gap Using Image Gradients for the Task of End-to-End Autonomous Drivingcited by 1
  2. 2022Leveraging Systematic Knowledge of 2D Transformationscited by 1