# Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery

arXiv: https://arxiv.org/abs/2406.06092 · HTML: https://mlsanity.com/paper/2406.06092/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real](https://mlsanity.com/paper/2006.09001/) (2020)
- [RetinaGAN: An Object-aware Approach to Sim-to-Real Transfer](https://mlsanity.com/paper/2011.03148/) (2020)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Safe Reinforcement Learning using Formal Verification for Tissue Retraction in Autonomous Robotic-Assisted Surgery](https://mlsanity.com/paper/2109.02323/) (2021)
- [Learning from Demonstrations for Autonomous Soft-tissue Retraction](https://mlsanity.com/paper/2110.00336/) (2021)
- [Autonomous Tissue Manipulation via Surgical Robot Using Learning Based Model Predictive Control](https://mlsanity.com/paper/1902.01459/) (2019)
- [Cooperative Assistance in Robotic Surgery through Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2110.04857/) (2021)
- [Self-supervised Transparent Liquid Segmentation for Robotic Pouring](https://mlsanity.com/paper/2203.01538/) (2022)
- [Synthetic-to-Real Domain Adaptation using Contrastive Unpaired Translation](https://mlsanity.com/paper/2203.09454/) (2022)
- [Deliberation in autonomous robotic surgery: a framework for handling anatomical uncertainty](https://mlsanity.com/paper/2203.05438/) (2022)

## Built on by (most cited first, 26 of 26)

- [Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/2503.24361/) (2025)
- [SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning](https://mlsanity.com/paper/2505.10251/) (2025)
- [Surgical Vision World Model](https://mlsanity.com/paper/2503.02904/) (2025)
- [SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation](https://mlsanity.com/paper/2603.13024/) (2026)
- [LUDO: Low-Latency Understanding of Deformable Objects using Point Cloud Occupancy Functions](https://mlsanity.com/paper/2411.08777/) (2024)
- [Grasping Deformable Objects via Reinforcement Learning with Cross-Modal Attention to Visuo-Tactile Inputs](https://mlsanity.com/paper/2504.15595/) (2025)
- [DOFS: A Real-world 3D Deformable Object Dataset with Full Spatial Information for Dynamics Model Learning](https://mlsanity.com/paper/2410.21758/) (2024)
- [MaNGO - Adaptable Graph Network Simulators via Meta-Learning](https://mlsanity.com/paper/2510.05874/) (2025)
- [Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning](https://mlsanity.com/paper/2409.02724/) (2024)
- [FF-SRL: High Performance GPU-Based Surgical Simulation For Robot Learning](https://mlsanity.com/paper/2503.18616/) (2025)
- [Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning](https://mlsanity.com/paper/2411.14622/) (2024)
- [CLASP: General-Purpose Clothes Manipulation with Semantic Keypoints](https://mlsanity.com/paper/2507.19983/) (2025)
- [Visuomotor Grasping with World Models for Surgical Robots](https://mlsanity.com/paper/2508.11200/) (2025)
- [Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning](https://mlsanity.com/paper/2510.25405/) (2025)
- [Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning](https://mlsanity.com/paper/2511.05234/) (2025)
- [NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation](https://mlsanity.com/paper/2605.15559/) (2026)
- [Embedded Image-to-Image Translation for Efficient Sim-to-Real Transfer in Learning-based Robot-Assisted Soft Manipulation](https://mlsanity.com/paper/2409.10204/) (2024)
- [SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learning](https://mlsanity.com/paper/2409.15651/) (2024)
- [Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency](https://mlsanity.com/paper/2506.19388/) (2025)
- [NeeCo: Image Synthesis of Novel Instrument States Based on Dynamic and Deformable 3D Gaussian Reconstruction](https://mlsanity.com/paper/2508.07897/) (2025)
- [A Koopman-Bayesian Framework for High-Fidelity, Perceptually Optimized Haptic Surgical Simulation](https://mlsanity.com/paper/2602.15834/) (2026)
- [AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction](https://mlsanity.com/paper/2604.01371/) (2026)
- [SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing](https://mlsanity.com/paper/2607.04616/) (2026)
- [Surg-UniWorld: A Unified Surgical World Model with Multimodal Control Experts](https://mlsanity.com/paper/2608.06770/) (2026)
- [VERAGMIL: Virtual Environment for Scooping Granular Foods with Imitation Learning Models](https://mlsanity.com/paper/2608.18258/) (2026)
- [Occupancy Network-Guided Autonomous Robotic Partial Nephrectomy](https://mlsanity.com/paper/2609.16186/) (2026)

## Abstract (arXiv)

Automation holds the potential to assist surgeons in robotic interventions, shifting their mental work load from visuomotor control to high level decision making. Reinforcement learning has shown promising results in learning complex visuomotor policies, especially in simulation environments where many samples can be collected at low cost. A core challenge is learning policies in simulation that can be deployed in the real world, thereby overcoming the sim-to-real gap. In this work, we bridge the visual sim-to-real gap with an image-based reinforcement learning pipeline based on pixel-level domain adaptation and demonstrate its effectiveness on an image-based task in deformable object manipulation. We choose a tissue retraction task because of its importance in clinical reality of precise cancer surgery. After training in simulation on domain-translated images, our policy requires no retraining to perform tissue retraction with a 50% success rate on the real robotic system using raw RGB images. Furthermore, our sim-to-real transfer method makes no assumptions on the task itself and requires no paired images. This work introduces the first successful application of visual sim-to-real transfer for robotic manipulation of deformable objects in the surgical field, which represents a notable step towards the clinical translation of cognitive surgical robotics.