# Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization

arXiv: https://arxiv.org/abs/2002.11635 · HTML: https://mlsanity.com/paper/2002.11635/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 30 of 47)

- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Polybot: Training One Policy Across Robots While Embracing Variability](https://mlsanity.com/paper/2307.03719/) (2023)
- [CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation](https://mlsanity.com/paper/2402.14795/) (2024)
- [Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing](https://mlsanity.com/paper/2103.04909/) (2021)
- [Natural Language Can Help Bridge the Sim2Real Gap](https://mlsanity.com/paper/2405.10020/) (2024)
- [Large Sequence Models for Sequential Decision-Making: A Survey](https://mlsanity.com/paper/2306.13945/) (2023)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real](https://mlsanity.com/paper/2203.11653/) (2022)
- [Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks](https://mlsanity.com/paper/2206.06282/) (2022)
- [SSRL: Self-Search Reinforcement Learning](https://mlsanity.com/paper/2508.10874/) (2025)
- [What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?](https://mlsanity.com/paper/2310.02219/) (2023)
- [High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures](https://mlsanity.com/paper/2504.17857/) (2025)
- [Continual Domain Randomization](https://mlsanity.com/paper/2403.12193/) (2024)
- [Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2404.16879/) (2024)
- [Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning](https://mlsanity.com/paper/2105.14780/) (2021)
- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems](https://mlsanity.com/paper/2403.15947/) (2024)
- [Pay Attention to How You Drive: Safe and Adaptive Model-Based Reinforcement Learning for Off-Road Driving](https://mlsanity.com/paper/2310.08674/) (2023)
- [Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviors](https://mlsanity.com/paper/2103.03793/) (2021)
- [A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning](https://mlsanity.com/paper/2310.08836/) (2023)
- [NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2403.02274/) (2024)
- [Multi-Fidelity Reinforcement Learning for Time-Optimal Quadrotor Re-planning](https://mlsanity.com/paper/2403.08152/) (2024)
- [Improving Offline Reinforcement Learning with Inaccurate Simulators](https://mlsanity.com/paper/2405.04307/) (2024)
- [Imitation learning for variable speed motion generation over multiple actions](https://mlsanity.com/paper/2103.06466/) (2021)
- [End-to-End Reinforcement Learning for Torque Based Variable Height Hopping](https://mlsanity.com/paper/2307.16676/) (2023)
- [Accelerating Robot Learning of Contact-Rich Manipulations: A Curriculum Learning Study](https://mlsanity.com/paper/2204.12844/) (2022)
- [Sim-To-Real Transfer of Visual Grounding for Human-Aided Ambiguity Resolution](https://mlsanity.com/paper/2205.12089/) (2022)

## Abstract (arXiv)

In this work we show how to use the Operational Space Control framework (OSC) under joint and cartesian constraints for reinforcement learning in cartesian space. Our method is therefore able to learn fast and with adjustable degrees of freedom, while we are able to transfer policies without additional dynamics randomizations on a KUKA LBR iiwa peg in-hole task. Before learning in simulation starts, we perform a system identification for aligning the simulation environment as far as possible with the dynamics of a real robot. Adding constraints to the OSC controller allows us to learn in a safe way on the real robot or to learn a flexible, goal conditioned policy that can be easily transferred from simulation to the real robot.