# DS3-Net: Difficulty-perceived Common-to-T1ce Semi-Supervised Multimodal MRI Synthesis Network

arXiv: https://arxiv.org/abs/2203.06920 · HTML: https://mlsanity.com/paper/2203.06920/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [MedGAN: Medical Image Translation using GANs](https://mlsanity.com/paper/1806.06397/) (2018)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [Brain Tumor Segmentation on MRI with Missing Modalities](https://mlsanity.com/paper/1904.07290/) (2019)
- [Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities](https://mlsanity.com/paper/2104.06231/) (2021)
- [Semi-Supervised Semantic Image Segmentation with Self-correcting Networks](https://mlsanity.com/paper/1811.07073/) (2018)
- [DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis](https://mlsanity.com/paper/1904.12894/) (2019)
- [mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis](https://mlsanity.com/paper/1909.11504/) (2019)
- [Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation](https://mlsanity.com/paper/2007.10787/) (2020)
- [Brain tumor segmentation with missing modalities via latent multi-source correlation representation](https://mlsanity.com/paper/2003.08870/) (2020)
- [Brain segmentation based on multi-atlas guided 3D fully convolutional network ensembles](https://mlsanity.com/paper/1901.01381/) (2019)
- [Fast T2w/FLAIR MRI Acquisition by Optimal Sampling of Information Complementary to Pre-acquired T1w MRI](https://mlsanity.com/paper/2111.06400/) (2021)
- [Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion](https://mlsanity.com/paper/2203.04586/) (2022)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation](https://mlsanity.com/paper/2403.20168/) (2024)

## Abstract (arXiv)

Contrast-enhanced T1 (T1ce) is one of the most essential magnetic resonance imaging (MRI) modalities for diagnosing and analyzing brain tumors, especially gliomas. In clinical practice, common MRI modalities such as T1, T2, and fluid attenuation inversion recovery are relatively easy to access while T1ce is more challenging considering the additional cost and potential risk of allergies to the contrast agent. Therefore, it is of great clinical necessity to develop a method to synthesize T1ce from other common modalities. Current paired image translation methods typically have the issue of requiring a large amount of paired data and do not focus on specific regions of interest, e.g., the tumor region, in the synthesization process. To address these issues, we propose a Difficulty-perceived common-to-T1ce Semi-Supervised multimodal MRI Synthesis network (DS3-Net), involving both paired and unpaired data together with dual-level knowledge distillation. DS3-Net predicts a difficulty map to progressively promote the synthesis task. Specifically, a pixelwise constraint and a patchwise contrastive constraint are guided by the predicted difficulty map. Through extensive experiments on the publiclyavailable BraTS2020 dataset, DS3-Net outperforms its supervised counterpart in each respect. Furthermore, with only 5% paired data, the proposed DS3-Net achieves competitive performance with state-of-theart image translation methods utilizing 100% paired data, delivering an average SSIM of 0.8947 and an average PSNR of 23.60.