# Zero-shot Sim2Real Adaptation Across Environments

arXiv: https://arxiv.org/abs/2302.04013 · HTML: https://mlsanity.com/paper/2302.04013/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Preparing for the Unknown: Learning a Universal Policy with Online System Identification](https://mlsanity.com/paper/1702.02453/) (2017)
- [EPOpt: Learning Robust Neural Network Policies Using Model Ensembles](https://mlsanity.com/paper/1610.01283/) (2016)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Auto-Tuned Sim-to-Real Transfer](https://mlsanity.com/paper/2104.07662/) (2021)
- [Data-efficient Domain Randomization with Bayesian Optimization](https://mlsanity.com/paper/2003.02471/) (2020)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing](https://mlsanity.com/paper/1808.03246/) (2018)
- [SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2101.06005/) (2021)
- [TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfer](https://mlsanity.com/paper/1907.11200/) (2019)
- [Combining Physical Simulators and Object-Based Networks for Control](https://mlsanity.com/paper/1904.06580/) (2019)
- [Reinforced Grounded Action Transformation for Sim-to-Real Transfer](https://mlsanity.com/paper/2008.01279/) (2020)
- [Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding](https://mlsanity.com/paper/2107.00339/) (2021)
- [Blending Diverse Physical Priors with Neural Networks](https://mlsanity.com/paper/1910.00201/) (2019)

## Built on by (most cited first, 1 of 1)

- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)

## Abstract (arXiv)

Simulation based learning often provides a cost-efficient recourse to reinforcement learning applications in robotics. However, simulators are generally incapable of accurately replicating real-world dynamics, and thus bridging the sim2real gap is an important problem in simulation based learning. Current solutions to bridge the sim2real gap involve hybrid simulators that are augmented with neural residual models. Unfortunately, they require a separate residual model for each individual environment configuration (i.e., a fixed setting of environment variables such as mass, friction etc.), and thus are not transferable to new environments quickly. To address this issue, we propose a Reverse Action Transformation (RAT) policy which learns to imitate simulated policies in the real-world. Once learnt from a single environment, RAT can then be deployed on top of a Universal Policy Network to achieve zero-shot adaptation to new environments. We empirically evaluate our approach in a set of continuous control tasks and observe its advantage as a few-shot and zero-shot learner over competing baselines.