# Leveraging Multimodal CycleGAN for the Generation of Anatomically Accurate Synthetic CT Scans from MRIs

arXiv: https://arxiv.org/abs/2407.10888 · HTML: https://mlsanity.com/paper/2407.10888/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Deep MR to CT Synthesis using Unpaired Data](https://mlsanity.com/paper/1708.01155/) (2017)
- [GANs for Medical Image Synthesis: An Empirical Study](https://mlsanity.com/paper/2105.05318/) (2021)
- [Unsupervised Medical Image Translation Using Cycle-MedGAN](https://mlsanity.com/paper/1903.03374/) (2019)
- [Deep CT to MR Synthesis using Paired and Unpaired Data](https://mlsanity.com/paper/1805.10790/) (2018)
- [Semi-supervised Task-driven Data Augmentation for Medical Image Segmentation](https://mlsanity.com/paper/2007.05363/) (2020)
- [Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets](https://mlsanity.com/paper/1901.11369/) (2019)
- [Comparative Study on Generative Adversarial Networks](https://mlsanity.com/paper/1801.04271/) (2018)
- [Semi-supervised mp-MRI Data Synthesis with StitchLayer and Auxiliary Distance Maximization](https://mlsanity.com/paper/1812.06625/) (2018)
- [Brain MRI Tumor Segmentation with Adversarial Networks](https://mlsanity.com/paper/1910.02717/) (2019)
- [Capturing Variabilities from Computed Tomography Images with Generative Adversarial Networks](https://mlsanity.com/paper/1805.11504/) (2018)

## Abstract (arXiv)

In many clinical settings, the use of both Computed Tomography (CT) and Magnetic Resonance (MRI) is necessary to pursue a thorough understanding of the patient's anatomy and to plan a suitable therapeutical strategy; this is often the case in MRI-based radiotherapy, where CT is always necessary to prepare the dose delivery, as it provides the essential information about the radiation absorption properties of the tissues. Sometimes, MRI is preferred to contour the target volumes. However, this approach is often not the most efficient, as it is more expensive, time-consuming and, most importantly, stressful for the patients. To overcome this issue, in this work, we analyse the capabilities of different configurations of Deep Learning models to generate synthetic CT scans from MRI, leveraging the power of Generative Adversarial Networks (GANs) and, in particular, the CycleGAN architecture, capable of working in an unsupervised manner and without paired images, which were not available. Several CycleGAN models were trained unsupervised to generate CT scans from different MRI modalities with and without contrast agents. To overcome the problem of not having a ground truth, distribution-based metrics were used to assess the model's performance quantitatively, together with a qualitative evaluation where physicians were asked to differentiate between real and synthetic images to understand how realistic the generated images were. The results show how, depending on the input modalities, the models can have very different performances; however, models with the best quantitative results, according to the distribution-based metrics used, can generate very difficult images to distinguish from the real ones, even for physicians, demonstrating the approach's potential.