mlsanityData to 7 Oct 2026

Auto-Tuned Sim-to-Real Transfer

arXiv 2104.07662 ↗First posted 15 Apr 2021cs.RO cs.AI cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Policies trained in simulation often fail when transferred to the real world due to the `reality gap' where the simulator is unable to accurately capture the dynamics and visual properties of the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering to determine how much to randomize system parameters in order to learn a policy that is robust to sim-to-real transfer while also not being too conservative. We propose a method for automatically tuning simulator system parameters to match the real world using only raw RGB images of the real world without the need to define rewards or estimate state. Our key insight is to reframe the auto-tuning of parameters as a search problem where we iteratively shift the simulation system parameters to approach the real-world system parameters. We propose a Search Param Model (SPM) that, given a sequence of observations and actions and a set of system parameters, predicts whether the given parameters are higher or lower than the true parameters used to generate the observations. We evaluate our method on multiple robotic control tasks in both sim-to-sim and sim-to-real transfer, demonstrating significant improvement over naive domain randomization. Project videos and code at https://yuqingd.github.io/autotuned-sim2real/

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

20 of 24 references.

References · 20

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2018DeepMind Control Suitecited by 935
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  9. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  10. 2020Reinforcement Learning with Augmented Datacited by 452
  11. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
8 more
  1. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  2. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  3. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  4. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
  5. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  6. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  7. 2019Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulationcited by 19
  8. 2020Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 12; 2023: 9; 2024: 7; 2025: 7; 2026: 5.

45 papers.

Sort:

By area:

  1. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  2. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  3. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  4. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  5. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  6. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 24
  7. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  8. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  9. 2021Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulationcited by 17
  10. 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 16
  11. 2023Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassemblycited by 16
  12. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
33 more
  1. 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gapscited by 15
  2. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 15
  3. 2025Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Datacited by 8
  4. 2024HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environmentscited by 6
  5. 2022Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filtercited by 5
  6. 2023Marginalized Importance Sampling for Off-Environment Policy Evaluationcited by 5
  7. 2023Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robotscited by 4
  8. 2024Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpointcited by 3
  9. 2023Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversitycited by 2
  10. 2023BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuningcited by 2
  11. 2024Simplex-enabled Safe Continual Learning Machinecited by 2
  12. 2025PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomizationcited by 2
  13. 2025Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filteringcited by 2
  14. 2021Single RGB-D Camera Teleoperation for General Robotic Manipulationcited by 1
  15. 2022Fast Model-based Policy Search for Universal Policy Networkscited by 1
  16. 2022Meta-simulation for the Automated Design of Synthetic Overhead Imagerycited by 1
  17. 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptationcited by 1
  18. 2023Zero-shot Sim2Real Adaptation Across Environmentscited by 1
  19. 2024Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methodscited by 1
  20. 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusioncited by 1
  21. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correctioncited by 1
  22. 2021Intuitive Physics Guided Exploration for Sample Efficient Sim2real Transfercited by 0
  23. 2021Adaptation of Quadruped Robot Locomotion with Meta-Learningcited by 0
  24. 2022Uncertainty Aware System Identification with Universal Policiescited by 0
  25. 2022Concurrent Policy Blending and System Identification for Generalized Assistive Controlcited by 0
  26. 2022Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learningcited by 0
  27. 2022Transferring Knowledge for Reinforcement Learning in Contact-Rich Manipulationcited by 0
  28. 2023PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptationcited by 0
  29. 2025Real-DRL: Teach and Learn in Realitycited by 0
  30. 2026Vid2Sid: Videos Can Help Close the Sim2Real Gapcited by 0
  31. 2026Cross-Domain Policy Optimization via Bellman Consistency and Hybrid Criticscited by 0
  32. 2026AI Sandboxes: A Threat Model, Taxonomy, and Measurement Frameworkcited by 0
  33. 2026How Should a Simulation-to-Reality Transfer Budget Be Spent?cited by 0