# Meta Reinforcement Learning for Sim-to-real Domain Adaptation

arXiv: https://arxiv.org/abs/1909.12906 · HTML: https://mlsanity.com/paper/1909.12906/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2019.

## Built on by (most cited first, 30 of 49)

- [Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data](https://mlsanity.com/paper/2412.07762/) (2024)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Lifelong Robotic Reinforcement Learning by Retaining Experiences](https://mlsanity.com/paper/2109.09180/) (2021)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [An In-depth Survey of Large Language Model-based Artificial Intelligence Agents](https://mlsanity.com/paper/2309.14365/) (2023)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Cross-Domain Policy Adaptation by Capturing Representation Mismatch](https://mlsanity.com/paper/2405.15369/) (2024)
- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms](https://mlsanity.com/paper/2103.03697/) (2021)
- [DROPO: Sim-to-Real Transfer with Offline Domain Randomization](https://mlsanity.com/paper/2201.08434/) (2022)
- [Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly](https://mlsanity.com/paper/2301.05033/) (2023)
- [Latent-Variable Advantage-Weighted Policy Optimization for Offline RL](https://mlsanity.com/paper/2203.08949/) (2022)
- [SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation](https://mlsanity.com/paper/2201.13248/) (2022)
- [Effective Tuning Strategies for Generalist Robot Manipulation Policies](https://mlsanity.com/paper/2410.01220/) (2024)
- [MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance](https://mlsanity.com/paper/2112.03575/) (2021)
- [Massively Digitized Power Grid: Opportunities and Challenges of Use-inspired AI](https://mlsanity.com/paper/2205.05180/) (2022)
- [Giving Feedback on Interactive Student Programs with Meta-Exploration](https://mlsanity.com/paper/2211.08802/) (2022)
- [Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning](https://mlsanity.com/paper/2502.02332/) (2025)
- [Solving the Real Robot Challenge using Deep Reinforcement Learning](https://mlsanity.com/paper/2109.15233/) (2021)
- [Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Tasks](https://mlsanity.com/paper/2205.09683/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [OMPO: A Unified Framework for RL under Policy and Dynamics Shifts](https://mlsanity.com/paper/2405.19080/) (2024)
- [Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution](https://mlsanity.com/paper/2412.13492/) (2024)
- [A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards](https://mlsanity.com/paper/2502.08643/) (2025)
- [Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents](https://mlsanity.com/paper/2203.08454/) (2022)
- [Variational Meta Reinforcement Learning for Social Robotics](https://mlsanity.com/paper/2206.03211/) (2022)
- [A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2312.15474/) (2023)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [From Abstraction to Reality: DARPA's Vision for Robust Sim-to-Real Autonomy](https://mlsanity.com/paper/2503.11007/) (2025)
- [The Emergence of Deep Reinforcement Learning for Path Planning](https://mlsanity.com/paper/2507.15469/) (2025)

## Abstract (arXiv)

Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.