# Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation

arXiv: https://arxiv.org/abs/2206.02679 · HTML: https://mlsanity.com/paper/2206.02679/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 6 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real](https://mlsanity.com/paper/2006.09001/) (2020)
- [Deep Reinforcement Learning for High Precision Assembly Tasks](https://mlsanity.com/paper/1708.04033/) (2017)
- [A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning](https://mlsanity.com/paper/1810.01531/) (2018)
- [Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks](https://mlsanity.com/paper/2004.14404/) (2020)
- [Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2008.10224/) (2020)
- [Affordance Learning for End-to-End Visuomotor Robot Control](https://mlsanity.com/paper/1903.04053/) (2019)
- [Adversarial Feature Training for Generalizable Robotic Visuomotor Control](https://mlsanity.com/paper/1909.07745/) (2019)

## Built on by (most cited first, 4 of 4)

- [Addressing Weak Decision Boundaries in Image Classification by Leveraging Web Search and Generative Models](https://mlsanity.com/paper/2310.19986/) (2023)
- [Extreme Image Transformations Facilitate Robust Latent Object Representations](https://mlsanity.com/paper/2310.07725/) (2023)
- [Reinforcement Learning for Robotic Insertion of Flexible Cables in Industrial Settings](https://mlsanity.com/paper/2509.13731/) (2025)
- [Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptation](https://mlsanity.com/paper/2601.16677/) (2026)

## Abstract (arXiv)

Reinforcement learning has shown a wide usage in robotics tasks, such as insertion and grasping. However, without a practical sim2real strategy, the policy trained in simulation could fail on the real task. There are also wide researches in the sim2real strategies, but most of those methods rely on heavy image rendering, domain randomization training, or tuning. In this work, we solve the insertion task using a pure visual reinforcement learning solution with minimum infrastructure requirement. We also propose a novel sim2real strategy, Real2Sim, which provides a novel and easier solution in policy adaptation. We discuss the advantage of Real2Sim compared with Sim2Real.