# Bi-Modality Medical Image Synthesis Using Semi-Supervised Sequential Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2308.14066 · HTML: https://mlsanity.com/paper/2308.14066/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 27 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Medical Image Synthesis with Context-Aware Generative Adversarial Networks](https://mlsanity.com/paper/1612.05362/) (2016)
- [Multi-Agent Diverse Generative Adversarial Networks](https://mlsanity.com/paper/1704.02906/) (2017)
- [XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation](https://mlsanity.com/paper/1904.09229/) (2019)
- [Unsupervised Learning for Cell-level Visual Representation in Histopathology Images with Generative Adversarial Networks](https://mlsanity.com/paper/1711.11317/) (2017)

## Built on by (most cited first, 3 of 3)

- [Semi-Supervised Disease Classification based on Limited Medical Image Data](https://mlsanity.com/paper/2405.04295/) (2024)
- [Label Filling via Mixed Supervision for Medical Image Segmentation from Noisy Annotations](https://mlsanity.com/paper/2410.16057/) (2024)
- [Towards Personalized Multi-Modal MRI Synthesis across Heterogeneous Datasets](https://mlsanity.com/paper/2602.19723/) (2026)

## Abstract (arXiv)

In this paper, we propose a bi-modality medical image synthesis approach based on sequential generative adversarial network (GAN) and semi-supervised learning. Our approach consists of two generative modules that synthesize images of the two modalities in a sequential order. A method for measuring the synthesis complexity is proposed to automatically determine the synthesis order in our sequential GAN. Images of the modality with a lower complexity are synthesized first, and the counterparts with a higher complexity are generated later. Our sequential GAN is trained end-to-end in a semi-supervised manner. In supervised training, the joint distribution of bi-modality images are learned from real paired images of the two modalities by explicitly minimizing the reconstruction losses between the real and synthetic images. To avoid overfitting limited training images, in unsupervised training, the marginal distribution of each modality is learned based on unpaired images by minimizing the Wasserstein distance between the distributions of real and fake images. We comprehensively evaluate the proposed model using two synthesis tasks based on three types of evaluate metrics and user studies. Visual and quantitative results demonstrate the superiority of our method to the state-of-the-art methods, and reasonable visual quality and clinical significance. Code is made publicly available at https://github.com/hustlinyi/Multimodal-Medical-Image-Synthesis.