mlsanityData to 14 Oct 2026

Asymmetric Actor Critic for Image-Based Robot Learning

arXiv 1710.06542 ↗First posted 18 Oct 2017cs.RO cs.AI cs.LGcited by 238

Not on the map. Why?

Cited by 91 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains. However, Robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator. While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator. In this work, we exploit the full state observability in the simulator to train better policies which take as input only partial observations (RGBD images). We do this by employing an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) gets rendered images as input. We show experimentally on a range of simulated tasks that using these asymmetric inputs significantly improves performance. Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block. We achieve this simulation to real world transfer without training on any real world data.

Key facts. Cited by 238 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 0; 2021: 32; 2022: 22; 2023: 21; 2024: 36; 2025: 55; 2026: 70.

238 papers.

Sort:

By area:

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  3. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  4. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  5. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  6. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  7. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Taskscited by 118
  8. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
  9. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  10. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
  11. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  12. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
38 more
  1. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 71
  2. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 71
  3. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  4. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
  5. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  6. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  7. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 56
  8. 2024Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learningcited by 56
  9. 2024Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfercited by 53
  10. 2021Auto-Tuned Sim-to-Real Transfercited by 49
  11. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  12. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  13. 2023Neural Volumetric Memory for Visual Locomotion Controlcited by 44
  14. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 44
  15. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  16. 2024Twisting Lids Off with Two Handscited by 42
  17. 2023Finetuning Offline World Models in the Real Worldcited by 41
  18. 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 38
  19. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 38
  20. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 36
  21. 2021Efficient Transformers in Reinforcement Learning using Actor-Learner Distillationcited by 35
  22. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 34
  23. 2025Self-Improving Embodied Foundation Modelscited by 32
  24. 2023Learning to Fly in Secondscited by 31
  25. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flightcited by 31
  26. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  27. 2021Robust Predictable Controlcited by 29
  28. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 27
  29. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  30. 2024Learning Humanoid Locomotion over Challenging Terraincited by 26
  31. 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 25
  32. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 24
  33. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  34. 2021A Generalizable Approach to Learning Optimizerscited by 23
  35. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 23
  36. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  37. 2022Learning Deformable Object Manipulation from Expert Demonstrationscited by 19
  38. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 19

50 most cited of 238 All 238 (JSON)