# SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning

arXiv: https://arxiv.org/abs/2101.06005 · HTML: https://mlsanity.com/paper/2101.06005/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2021.

## Built on by (most cited first, 27 of 27)

- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation](https://mlsanity.com/paper/2402.15399/) (2024)
- [What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2212.02705/) (2022)
- [ORV: 4D Occupancy-centric Robot Video Generation](https://mlsanity.com/paper/2506.03079/) (2025)
- [Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2409.20521/) (2024)
- [Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models](https://mlsanity.com/paper/2410.09163/) (2024)
- [Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2410.23450/) (2024)
- [Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective](https://mlsanity.com/paper/2305.18464/) (2023)
- [Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets](https://mlsanity.com/paper/2310.04517/) (2023)
- [Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots](https://mlsanity.com/paper/2303.09597/) (2023)
- [Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution](https://mlsanity.com/paper/2309.12529/) (2023)
- [Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control](https://mlsanity.com/paper/2307.12388/) (2023)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Facilitating Sim-to-real by Intrinsic Stochasticity of Real-Time Simulation in Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2304.06056/) (2023)
- [Autonomous Advanced Aerial Mobility -- An End-to-end Autonomy Framework for UAVs and Beyond](https://mlsanity.com/paper/2311.04472/) (2023)
- [Zero-shot Sim2Real Adaptation Across Environments](https://mlsanity.com/paper/2302.04013/) (2023)
- [Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Picking](https://mlsanity.com/paper/2504.15414/) (2025)
- [Error-Aware Policy Learning: Zero-Shot Generalization in Partially Observable Dynamic Environments](https://mlsanity.com/paper/2103.07732/) (2021)
- [Unified Simulation, Perception, and Generation of Human Behavior](https://mlsanity.com/paper/2204.13678/) (2022)
- [Off-dynamics Conditional Diffusion Planners](https://mlsanity.com/paper/2410.12238/) (2024)
- [Dual Action Policy for Robust Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2410.12250/) (2024)
- [Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptation](https://mlsanity.com/paper/2601.16677/) (2026)

## Abstract (arXiv)

As learning-based approaches progress towards automating robot controllers design, transferring learned policies to new domains with different dynamics (e.g. sim-to-real transfer) still demands manual effort. This paper introduces SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics simulator to match the simulated trajectories to the ones from the target domain, using a learned discriminative loss to address the limitations associated with manual loss design. Our hybrid simulator combines neural networks and traditional physics simulation to balance expressiveness and generalizability, and alleviates the need for a carefully selected parameter set in System ID. Once the hybrid simulator is identified via adversarial reinforcement learning, it can be used to refine policies for the target domain, without the need to interleave data collection and policy refinement. We show that our approach outperforms multiple strong baselines on six robotic locomotion tasks for domain adaptation.