mlsanityData to 7 Oct 2026

CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis

arXiv 2303.14081 ↗First posted 24 Mar 2023eess.IV cs.CVcited by 64

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

MRI synthesis promises to mitigate the challenge of missing MRI modality in clinical practice. Diffusion model has emerged as an effective technique for image synthesis by modelling complex and variable data distributions. However, most diffusion-based MRI synthesis models are using a single modality. As they operate in the original image domain, they are memory-intensive and less feasible for multi-modal synthesis. Moreover, they often fail to preserve the anatomical structure in MRI. Further, balancing the multiple conditions from multi-modal MRI inputs is crucial for multi-modal synthesis. Here, we propose the first diffusion-based multi-modality MRI synthesis model, namely Conditioned Latent Diffusion Model (CoLa-Diff). To reduce memory consumption, we design CoLa-Diff to operate in the latent space. We propose a novel network architecture, e.g., similar cooperative filtering, to solve the possible compression and noise in latent space. To better maintain the anatomical structure, brain region masks are introduced as the priors of density distributions to guide diffusion process. We further present auto-weight adaptation to employ multi-modal information effectively. Our experiments demonstrate that CoLa-Diff outperforms other state-of-the-art MRI synthesis methods, promising to serve as an effective tool for multi-modal MRI synthesis.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 18 of its 32 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

18 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Models17 Jul

Other references · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  6. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  7. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  8. 2019Seeing What a GAN Cannot Generatecited by 148
  9. 2020Attentional Feature Fusioncited by 115
  10. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  11. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  12. 2022Conversion Between CT and MRI Images Using Diffusion and Score-Matching Modelscited by 69
5 more
  1. 2019DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesiscited by 36
  2. 2019A Closer Look at the Optimization Landscapes of Generative Adversarial Networkscited by 25
  3. 2022Multi-modal learning for predicting the genotype of gliomacited by 15
  4. 2020Progressively Volumetrized Deep Generative Models for Data-Efficient Contextual Learning of MR Image Recoverycited by 14
  5. 2022MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Braincited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 22; 2025: 26; 2026: 12.

64 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  2. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  3. 2024Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolutioncited by 29
  4. 2023DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Modelcited by 23
  5. 2025What You Have is What You Track: Adaptive and Robust Multimodal Trackingcited by 15
  6. 2024Generative Enhancement for 3D Medical Imagescited by 14
  7. 2025Metrics that matter: Evaluating image quality metrics for medical image generationcited by 12
  8. 2024Advancing Text-Driven Chest X-Ray Generation with Policy-Based Reinforcement Learningcited by 10
  9. 2023A Survey of Emerging Applications of Diffusion Probabilistic Models in MRIcited by 9
  10. 2024Phy-Diff: Physics-guided Hourglass Diffusion Model for Diffusion MRI Synthesiscited by 8
  11. 2024Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Expertscited by 8
  12. 2024SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Modelscited by 8
38 more
  1. 2025Regression is all you need for medical image translationcited by 8
  2. 2025TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiencycited by 8
  3. 20243D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimescited by 6
  4. 2023Shape-guided Conditional Latent Diffusion Models for Synthesising Brain Vasculaturecited by 5
  5. 2025Diffusion Models for Computational Neuroimaging: A Surveycited by 5
  6. 2023A Unified Framework for Generative Data Augmentation: A Comprehensive Surveycited by 4
  7. 2024Unpaired Volumetric Harmonization of Brain MRI with Conditional Latent Diffusioncited by 4
  8. 2025FetalFlex: Anatomy-Guided Diffusion Model for Flexible Control on Fetal Ultrasound Image Synthesiscited by 4
  9. 2025Angio-Diff: Learning a Self-Supervised Adversarial Diffusion Model for Angiographic Geometry Generationcited by 4
  10. 2025LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesiscited by 4
  11. 2024CAVM: Conditional Autoregressive Vision Model for Contrast-Enhanced Brain Tumor MRI Synthesiscited by 3
  12. 2025FgC2F-UDiff: Frequency-guided and Coarse-to-fine Unified Diffusion Model for Multi-modality Missing MRI Synthesiscited by 3
  13. 20253D Wavelet Latent Diffusion Model for Whole-Body MR-to-CT Modality Translationcited by 3
  14. 2024MCAD: Multi-modal Conditioned Adversarial Diffusion Model for High-Quality PET Image Reconstructioncited by 2
  15. 2024I2I-Galip: Unsupervised Medical Image Translation Using Generative Adversarial CLIPcited by 2
  16. 2024HC3L-Diff: Hybrid conditional latent diffusion with high frequency enhancement for CBCT-to-CT synthesiscited by 2
  17. 2024Sub-DM:Subspace Diffusion Model with Orthogonal Decomposition for MRI Reconstructioncited by 2
  18. 2024Artificial Intelligence for Biomedical Video Generationcited by 2
  19. 2026TumorFlow: Physics-Guided Longitudinal MRI Synthesis of Glioblastoma Growthcited by 2
  20. 2024Pseudo-MRI-Guided PET Image Reconstruction Method Based on a Diffusion Probabilistic Modelcited by 1
  21. 2024FetalDiffusion: Pose-Controllable 3D Fetal MRI Synthesis with Conditional Diffusion Modelcited by 1
  22. 2024Temporal Evolution of Knee Osteoarthritis: A Diffusion-based Morphing Model for X-ray Medical Image Synthesiscited by 1
  23. 2025DAMM-Diffusion: Learning Divergence-Aware Multi-Modal Diffusion Model for Nanoparticles Distribution Predictioncited by 1
  24. 2025AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysiscited by 1
  25. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Reviewcited by 1
  26. 2025Tumor Synthesis conditioned on Radiomicscited by 1
  27. 2025FS-RWKV: Leveraging Frequency Spatial-Aware RWKV for 3T-to-7T MRI Translationcited by 1
  28. 2025MRI-to-CT Synthesis With Cranial Suture Segmentations Using A Variational Autoencoder Frameworkcited by 1
  29. 2026A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Modelscited by 1
  30. 2026Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridgecited by 1
  31. 2024Mitigating analytical variability in fMRI results with style transfercited by 0
  32. 2024Contrastive-Adversarial and Diffusion: Exploring pre-training and fine-tuning strategies for sulcal identificationcited by 0
  33. 2024Diffusion Model-based FOD Restoration from High Distortion in dMRIcited by 0
  34. 2025Fully Guided Neural Schr"odinger bridge for Brain MR image synthesiscited by 0
  35. 2025ZECO: ZeroFusion Guided 3D MRI Conditional Generationcited by 0
  36. 2025HepatoGEN: Generating Hepatobiliary Phase MRI with Perceptual and Adversarial Modelscited by 0
  37. 2025GM-LDM: Latent Diffusion Model for Brain Biomarker Identification through Functional Data-Driven Gray Matter Synthesiscited by 0
  38. 2025MultiViT2: A Data-augmented Multimodal Neuroimaging Prediction Framework via Latent Diffusion Modelcited by 0

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis
  2. 2026Multiscale Structure-Guided Latent Diffusion for Multimodal MRI Translation
  3. 2025AMM-Diff: Adaptive Multi-Modality Diffusion Network for Missing Modality Imputation
  4. 2025D2Diff : A Dual Domain Diffusion Model for Accurate Multi-Contrast MRI Synthesis
  5. 2025SLaM-DiMM: Shared Latent Modeling for Diffusion Based Missing Modality Synthesis in MRI
  6. 2022Adaptive Diffusion Priors for Accelerated MRI Reconstruction
  7. 2025MDPG: Multi-domain Diffusion Prior Guidance for MRI Reconstruction
  8. 2024McCaD: Multi-Contrast MRI Conditioned, Adaptive Adversarial Diffusion Model for High-Fidelity MRI Synthesis