# Realistic Endoscopic Image Generation Method Using Virtual-to-real Image-domain Translation

arXiv: https://arxiv.org/abs/2201.04918 · HTML: https://mlsanity.com/paper/2201.04918/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 13 Jan 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [Deep MR to CT Synthesis using Unpaired Data](https://mlsanity.com/paper/1708.01155/) (2017)
- [Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data size](https://mlsanity.com/paper/1803.06629/) (2018)
- [Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truth](https://mlsanity.com/paper/1712.07695/) (2017)
- [Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries](https://mlsanity.com/paper/1806.03627/) (2018)

## Built on by (most cited first, 1 of 1)

- [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024)

## Abstract (arXiv)

This paper proposes a realistic image generation method for visualization in endoscopic simulation systems. Endoscopic diagnosis and treatment are performed in many hospitals. To reduce complications related to endoscope insertions, endoscopic simulation systems are used for training or rehearsal of endoscope insertions. However, current simulation systems generate non-realistic virtual endoscopic images. To improve the value of the simulation systems, improvement of reality of their generated images is necessary. We propose a realistic image generation method for endoscopic simulation systems. Virtual endoscopic images are generated by using a volume rendering method from a CT volume of a patient. We improve the reality of the virtual endoscopic images using a virtual-to-real image-domain translation technique. The image-domain translator is implemented as a fully convolutional network (FCN). We train the FCN by minimizing a cycle consistency loss function. The FCN is trained using unpaired virtual and real endoscopic images. To obtain high quality image-domain translation results, we perform an image cleansing to the real endoscopic image set. We tested to use the shallow U-Net, U-Net, deep U-Net, and U-Net having residual units as the image-domain translator. The deep U-Net and U-Net having residual units generated quite realistic images.