mlsanityData to 14 Oct 2026

AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer

arXiv 2302.04903 ↗First posted 9 Feb 2023cs.RO cs.LG cs.SYcited by 18

Not on the map. Why?

Abstract (arXiv)

Simulation parameter settings such as contact models and object geometry approximations are critical to training robust robotic policies capable of transferring from simulation to real-world deployment. Previous approaches typically handcraft distributions over such parameters (domain randomization), or identify parameters that best match the dynamics of the real environment (system identification). However, there is often an irreducible gap between simulation and reality: attempting to match the dynamics between simulation and reality across all states and tasks may be infeasible and may not lead to policies that perform well in reality for a specific task. Addressing this issue, we propose AdaptSim, a new task-driven adaptation framework for sim-to-real transfer that aims to optimize task performance in target (real) environments -- instead of matching dynamics between simulation and reality. First, we meta-learn an adaptation policy in simulation using reinforcement learning for adjusting the simulation parameter distribution based on the current policy's performance in a target environment. We then perform iterative real-world adaptation by inferring new simulation parameter distributions for policy training, using a small amount of real data. We perform experiments in three robotic tasks: (1) swing-up of linearized double pendulum, (2) dynamic table-top pushing of a bottle, and (3) dynamic scooping of food pieces with a spatula. Our extensive simulation and hardware experiments demonstrate AdaptSim achieving 1-3x asymptotic performance and ~2x real data efficiency when adapting to different environments, compared to methods based on Sys-ID and directly training the task policy in target environments. Website: https://irom-lab.github.io/AdaptSim/

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 39 of its 49 references that are in the map. First posted on arXiv 9 Feb 2023.

Builds on

39 of 49 references.

References · 39

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  8. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  9. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  10. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  11. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  12. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
27 more
  1. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  2. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  3. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  4. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
  5. 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 85
  6. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
  7. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  8. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
  9. 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 47
  10. 2018Learning To Simulatecited by 44
  11. 2018Policy Transfer with Strategy Optimizationcited by 43
  12. 2022Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objectscited by 37
  13. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
  14. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 27
  15. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  16. 2019How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?cited by 25
  17. 2019TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfercited by 22
  18. 2019Combining Physical Simulators and Object-Based Networks for Controlcited by 18
  19. 2018A Framework for Robot Manipulation: Skill Formalism, Meta Learning and Adaptive Controlcited by 17
  20. 2021Distributionally Robust Policy Learning via Adversarial Environment Generationcited by 17
  21. 2021Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulationcited by 17
  22. 2021Meta Learning Black-Box Population-Based Optimizerscited by 13
  23. 2020Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gapcited by 11
  24. 2021A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulationcited by 11
  25. 2021Velocity Level Approximation of Pressure Field Contact Patchescited by 5
  26. 2021BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with IsaacGymcited by 3
  27. 2022Task-Driven Hybrid Model Reduction for Dexterous Manipulationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024Diffusion Policy Policy Optimizationcited by 280
  2. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
  3. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  4. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 46
  5. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  6. 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 28
  7. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  8. 2025Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twincited by 12
  9. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  10. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoidscited by 7
  11. 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 4
  12. 2026D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Graspingcited by 3
6 more
  1. 2025Neural Fidelity Calibration for Informative Sim-to-Real Adaptationcited by 1
  2. 2025Data-Driven Dynamic Parameter Learning of manipulator robotscited by 1
  3. 2025Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approachcited by 0
  4. 2025Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulationcited by 0
  5. 2026How Should a Simulation-to-Reality Transfer Budget Be Spent?cited by 0
  6. 2026Coverage Aware Active Evaluation for Failure Discovery with Paired Systemscited by 0