# Bridging the gap between paired and unpaired medical image translation

arXiv: https://arxiv.org/abs/2110.08407 · HTML: https://mlsanity.com/paper/2110.08407/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [MedGAN: Medical Image Translation using GANs](https://mlsanity.com/paper/1806.06397/) (2018)
- [Distribution Matching Losses Can Hallucinate Features in Medical Image Translation](https://mlsanity.com/paper/1805.08841/) (2018)
- [Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network](https://mlsanity.com/paper/1802.09655/) (2018)
- [Medical Image Synthesis with Context-Aware Generative Adversarial Networks](https://mlsanity.com/paper/1612.05362/) (2016)
- [Deep MR to CT Synthesis using Unpaired Data](https://mlsanity.com/paper/1708.01155/) (2017)
- [Unsupervised Medical Image Translation Using Cycle-MedGAN](https://mlsanity.com/paper/1903.03374/) (2019)
- [Deep CT to MR Synthesis using Paired and Unpaired Data](https://mlsanity.com/paper/1805.10790/) (2018)
- [Generative Adversarial Networks for Image-to-Image Translation on Multi-Contrast MR Images - A Comparison of CycleGAN and UNIT](https://mlsanity.com/paper/1806.07777/) (2018)
- [PSIGAN: Joint probabilistic segmentation and image distribution matching for unpaired cross-modality adaptation based MRI segmentation](https://mlsanity.com/paper/2007.09465/) (2020)
- [Generative Adversarial Networks for MR-CT Deformable Image Registration](https://mlsanity.com/paper/1807.07349/) (2018)

## Built on by (most cited first, 6 of 6)

- [Unpaired Optical Coherence Tomography Angiography Image Super-Resolution via Frequency-Aware Inverse-Consistency GAN](https://mlsanity.com/paper/2309.17269/) (2023)
- [Translating Simulation Images to X-ray Images via Multi-Scale Semantic Matching](https://mlsanity.com/paper/2304.07693/) (2023)
- [StyleX: A Trainable Metric for X-ray Style Distances](https://mlsanity.com/paper/2405.14718/) (2024)
- [Repeat and Concatenate: 2D to 3D Image Translation with 3D to 3D Generative Modeling](https://mlsanity.com/paper/2406.18422/) (2024)
- [Image-to-Image Translation Based on Deep Generative Modeling for Radiotherapy Synthetic Dataset Creation](https://mlsanity.com/paper/2410.01828/) (2024)
- [Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization](https://mlsanity.com/paper/2410.02628/) (2024)

## Abstract (arXiv)

Medical image translation has the potential to reduce the imaging workload, by removing the need to capture some sequences, and to reduce the annotation burden for developing machine learning methods. GANs have been used successfully to translate images from one domain to another, such as MR to CT. At present, paired data (registered MR and CT images) or extra supervision (e.g. segmentation masks) is needed to learn good translation models. Registering multiple modalities or annotating structures within each of them is a tedious and laborious task. Thus, there is a need to develop improved translation methods for unpaired data. Here, we introduce modified pix2pix models for tasks CTrightarrowMR and MRrightarrowCT, trained with unpaired CT and MR data, and MRCAT pairs generated from the MR scans. The proposed modifications utilize the paired MR and MRCAT images to ensure good alignment between input and translated images, and unpaired CT images ensure the MRrightarrowCT model produces realistic-looking CT and CTrightarrowMR model works well with real CT as input. The proposed pix2pix variants outperform baseline pix2pix, pix2pixHD and CycleGAN in terms of FID and KID, and generate more realistic looking CT and MR translations.