# Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation

arXiv: https://arxiv.org/abs/1907.02882 · HTML: https://mlsanity.com/paper/1907.02882/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 30 of 38)

- [Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures](https://mlsanity.com/paper/2307.15220/) (2023)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Latent Graph Representations for Critical View of Safety Assessment](https://mlsanity.com/paper/2212.04155/) (2022)
- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Data](https://mlsanity.com/paper/2103.17204/) (2021)
- [Investigating Data Memorization in 3D Latent Diffusion Models for Medical Image Synthesis](https://mlsanity.com/paper/2307.01148/) (2023)
- [Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art](https://mlsanity.com/paper/2304.13014/) (2023)
- [SimT: Handling Open-set Noise for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.15202/) (2022)
- [Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation](https://mlsanity.com/paper/2107.06941/) (2021)
- [One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video](https://mlsanity.com/paper/2103.12988/) (2021)
- [Synthesising Rare Cataract Surgery Samples with Guided Diffusion Models](https://mlsanity.com/paper/2308.02587/) (2023)
- [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025)
- [VidLPRO: A underlineVideo-underlineLanguage underlinePre-training Framework for underlineRobotic and Laparoscopic Surgery](https://mlsanity.com/paper/2409.04732/) (2024)
- [Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation](https://mlsanity.com/paper/2103.01593/) (2021)
- [Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications](https://mlsanity.com/paper/2309.03048/) (2023)
- [Interactive Generation of Laparoscopic Videos with Diffusion Models](https://mlsanity.com/paper/2406.06537/) (2024)
- [Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models](https://mlsanity.com/paper/2410.07753/) (2024)
- [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries](https://mlsanity.com/paper/2110.12555/) (2021)
- [Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation](https://mlsanity.com/paper/2312.03043/) (2023)
- [Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection](https://mlsanity.com/paper/2504.15152/) (2025)
- [Surgical Text-to-Image Generation](https://mlsanity.com/paper/2407.09230/) (2024)
- [CryoGEM: Physics-Informed Generative Cryo-Electron Microscopy](https://mlsanity.com/paper/2312.02235/) (2023)
- [Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery](https://mlsanity.com/paper/2402.05860/) (2024)
- [SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models](https://mlsanity.com/paper/2408.09822/) (2024)
- [Neural Neural Textures Make Sim2Real Consistent](https://mlsanity.com/paper/2206.13500/) (2022)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [MeshBrush: Painting the Anatomical Mesh with Neural Stylization for Endoscopy](https://mlsanity.com/paper/2404.02999/) (2024)

## Abstract (arXiv)

In the medical domain, the lack of large training data sets and benchmarks is often a limiting factor for training deep neural networks. In contrast to expensive manual labeling, computer simulations can generate large and fully labeled data sets with a minimum of manual effort. However, models that are trained on simulated data usually do not translate well to real scenarios. To bridge the domain gap between simulated and real laparoscopic images, we exploit recent advances in unpaired image-to-image translation. We extent an image-to-image translation method to generate a diverse multitude of realistically looking synthetic images based on images from a simple laparoscopy simulation. By incorporating means to ensure that the image content is preserved during the translation process, we ensure that the labels given for the simulated images remain valid for their realistically looking translations. This way, we are able to generate a large, fully labeled synthetic data set of laparoscopic images with realistic appearance. We show that this data set can be used to train models for the task of liver segmentation of laparoscopic images. We achieve average dice scores of up to 0.89 in some patients without manually labeling a single laparoscopic image and show that using our synthetic data to pre-train models can greatly improve their performance. The synthetic data set will be made publicly available, fully labeled with segmentation maps, depth maps, normal maps, and positions of tools and camera (http://opencas.dkfz.de/image2image).