# A Novel Unified Conditional Score-based Generative Framework for Multi-modal Medical Image Completion

arXiv: https://arxiv.org/abs/2207.03430 · HTML: https://mlsanity.com/paper/2207.03430/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 16 of 16)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis](https://mlsanity.com/paper/2002.05000/) (2020)
- [LOGAN: Latent Optimisation for Generative Adversarial Networks](https://mlsanity.com/paper/1912.00953/) (2019)
- [A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translation](https://mlsanity.com/paper/2101.05434/) (2021)
- [A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translation](https://mlsanity.com/paper/2107.11945/) (2021)
- [Medical Image Imputation from Image Collections](https://mlsanity.com/paper/1808.05732/) (2018)

## Built on by (most cited first, 23 of 23)

- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024)
- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [Self-Consistent Recursive Diffusion Bridge for Medical Image Translation](https://mlsanity.com/paper/2405.06789/) (2024)
- [Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey](https://mlsanity.com/paper/2408.12880/) (2024)
- [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025)
- [Generative AI-Driven Human Digital Twin in IoT-Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2401.13699/) (2024)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts](https://mlsanity.com/paper/2410.19378/) (2024)
- [Diff-Def: Diffusion-Generated Deformation Fields for Conditional Atlases](https://mlsanity.com/paper/2403.16776/) (2024)
- [AMM-Diff: Adaptive Multi-Modality Diffusion Network for Missing Modality Imputation](https://mlsanity.com/paper/2501.12840/) (2025)
- [Cas-DiffCom: Cascaded diffusion model for infant longitudinal super-resolution 3D medical image completion](https://mlsanity.com/paper/2402.13776/) (2024)
- [McCaD: Multi-Contrast MRI Conditioned, Adaptive Adversarial Diffusion Model for High-Fidelity MRI Synthesis](https://mlsanity.com/paper/2409.00585/) (2024)
- [Score-based Generative Models for Photoacoustic Image Reconstruction with Rotation Consistency Constraints](https://mlsanity.com/paper/2306.13843/) (2023)
- [CT-SDM: A Sampling Diffusion Model for Sparse-View CT Reconstruction across All Sampling Rates](https://mlsanity.com/paper/2409.01571/) (2024)
- [QID2: An Image-Conditioned Diffusion Model for Q-space Up-sampling of DWI Data](https://mlsanity.com/paper/2409.02309/) (2024)
- [Multiple Noises in Diffusion Model for Semi-Supervised Multi-Domain Translation](https://mlsanity.com/paper/2309.14394/) (2023)
- [Synthesizing Missing MRI Sequences from Available Modalities using Generative Adversarial Networks in BraTS Dataset](https://mlsanity.com/paper/2310.07250/) (2023)
- [HepatoGEN: Generating Hepatobiliary Phase MRI with Perceptual and Adversarial Models](https://mlsanity.com/paper/2504.18405/) (2025)

## Abstract (arXiv)

Multi-modal medical image completion has been extensively applied to alleviate the missing modality issue in a wealth of multi-modal diagnostic tasks. However, for most existing synthesis methods, their inferences of missing modalities can collapse into a deterministic mapping from the available ones, ignoring the uncertainties inherent in the cross-modal relationships. Here, we propose the Unified Multi-Modal Conditional Score-based Generative Model (UMM-CSGM) to take advantage of Score-based Generative Model (SGM) in modeling and stochastically sampling a target probability distribution, and further extend SGM to cross-modal conditional synthesis for various missing-modality configurations in a unified framework. Specifically, UMM-CSGM employs a novel multi-in multi-out Conditional Score Network (mm-CSN) to learn a comprehensive set of cross-modal conditional distributions via conditional diffusion and reverse generation in the complete modality space. In this way, the generation process can be accurately conditioned by all available information, and can fit all possible configurations of missing modalities in a single network. Experiments on BraTS19 dataset show that the UMM-CSGM can more reliably synthesize the heterogeneous enhancement and irregular area in tumor-induced lesions for any missing modalities.