# A Vessel-Segmentation-Based CycleGAN for Unpaired Multi-modal Retinal Image Synthesis

arXiv: https://arxiv.org/abs/2306.02901 · HTML: https://mlsanity.com/paper/2306.02901/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 11 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Builds on (most cited first, 5 of 5)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation](https://mlsanity.com/paper/2203.02557/) (2022)
- [VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers](https://mlsanity.com/paper/2104.06757/) (2021)
- [Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network](https://mlsanity.com/paper/2207.10506/) (2022)
- [Explainable Diabetic Retinopathy Detection and Retinal Image Generation](https://mlsanity.com/paper/2107.00296/) (2021)

## Built on by (most cited first, 1 of 1)

- [Learn2Synth: Learning Optimal Data Synthesis Using Hypergradients for Brain Image Segmentation](https://mlsanity.com/paper/2411.16719/) (2024)

## Abstract (arXiv)

Unpaired image-to-image translation of retinal images can efficiently increase the training dataset for deep-learning-based multi-modal retinal registration methods. Our method integrates a vessel segmentation network into the image-to-image translation task by extending the CycleGAN framework. The segmentation network is inserted prior to a UNet vision transformer generator network and serves as a shared representation between both domains. We reformulate the original identity loss to learn the direct mapping between the vessel segmentation and the real image. Additionally, we add a segmentation loss term to ensure shared vessel locations between fake and real images. In the experiments, our method shows a visually realistic look and preserves the vessel structures, which is a prerequisite for generating multi-modal training data for image registration.