mlsanityData to 14 Oct 2026

3D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-quality Medical Image Generation

arXiv 2412.13059 ↗First posted 17 Dec 2024eess.IV cs.CVcited by 37

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

The generation of medical images presents significant challenges due to their high-resolution and three-dimensional nature. Existing methods often yield suboptimal performance in generating high-quality 3D medical images, and there is currently no universal generative framework for medical imaging. In this paper, we introduce a 3D Medical Latent Diffusion (3D MedDiffusion) model for controllable, high-quality 3D medical image generation. 3D MedDiffusion incorporates a novel, highly efficient Patch-Volume Autoencoder that compresses medical images into latent space through patch-wise encoding and recovers back into image space through volume-wise decoding. Additionally, we design a new noise estimator to capture both local details and global structural information during diffusion denoising process. 3D MedDiffusion can generate fine-detailed, high-resolution images (up to 512x512x512) and effectively adapt to various downstream tasks as it is trained on large-scale datasets covering CT and MRI modalities and different anatomical regions (from head to leg). Experimental results demonstrate that 3D MedDiffusion surpasses state-of-the-art methods in generative quality and exhibits strong generalizability across tasks such as sparse-view CT reconstruction, fast MRI reconstruction, and data augmentation for segmentation and classification. Source code and checkpoints are available at https://github.com/ShanghaiTech-IMPACT/3D-MedDiffusion.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 50 of its 66 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

50 of 66 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024MAISI: Medical AI for Synthetic Imaging13 Sep
  2. 2024WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis29 Feb
  3. 2023Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study1 Nov
  4. 2023Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis19 Jul
  5. 2022Diffusion Probabilistic Models beat GANs on Medical Images14 Dec
  6. 2022DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction22 Nov
  7. 2022Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation7 Nov
  8. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images27 Sep
  9. 2022High-Frequency Space Diffusion Models for Accelerated MRI10 Aug
  10. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Models17 Jul
  11. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Models1 Dec

Other references · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2022Generative Adversarial Networkscited by 5,627
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  12. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
27 more
  1. 2015Variational Inference with Normalizing Flowscited by 2,170
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  4. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
  5. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
  6. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 433
  7. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  8. 2018nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentationcited by 291
  9. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 288
  10. 2020AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?cited by 224
  11. 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 172
  12. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 132
  13. 2016Mode Regularized Generative Adversarial Networkscited by 123
  14. 2020MosMedData: Chest CT Scans With COVID-19 Related Findings Datasetcited by 104
  15. 2020Hierarchical Amortized Training for Memory-efficient High Resolution 3D GANcited by 60
  16. 2021CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomographycited by 46
  17. 2023MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Imagescited by 41
  18. 2020Alleviating Class-wise Gradient Imbalance for Pulmonary Airway Segmentationcited by 35
  19. 2023MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generationcited by 34
  20. 2019AirwayNet: A Voxel-Connectivity Aware Approach for Accurate Airway Segmentation Using Convolutional Neural Networkscited by 32
  21. 2020Vox2Vox: 3D-GAN for Brain Tumour Segmentationcited by 30
  22. 2024Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decodercited by 25
  23. 2018Development and Validation of Deep Learning Algorithms for Detection of Critical Findings in Head CT Scanscited by 24
  24. 2020Modelling the Distribution of 3D Brain MRI using a 2D Slice VAEcited by 20
  25. 2022BREAK: Bronchi Reconstruction by gEodesic transformation And sKeleton embeddingcited by 16
  26. 2024Deep Generative Models for 3D Medical Image Synthesiscited by 14
  27. 2016Discriminative Regularization for Generative Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 27.

37 papers.

Sort:

By area:

  1. 2025MAISI-v2: Accelerated 3D High-Resolution Medical Image Synthesis with Rectified Flow and Region-specific Contrastive Losscited by 20
  2. 2025Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translationcited by 8
  3. 2025Applications of Large Models in Medicinecited by 6
  4. 2025Latent Diffusion Autoencoders: Toward Efficient and Meaningful Unsupervised Representation Learning in Medical Imagingcited by 4
  5. 2026Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticitycited by 3
  6. 2026Scalable next-scale autoregression for medical image generation across anatomical regionscited by 3
  7. 2026TumorFlow: Physics-Guided Longitudinal MRI Synthesis of Glioblastoma Growthcited by 2
  8. 2026GenMed: A Pairwise Generative Reformulation of Medical Diagnostic Taskscited by 2
  9. 2025SynBT: High-quality Tumor Synthesis for Breast Tumor Segmentation by 3D Diffusion Modelcited by 1
  10. 2026MeDUET: Disentangled Unified Pretraining for 3D Medical Image Synthesis and Analysiscited by 1
  11. 2026Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesiscited by 1
  12. 2026GloResNet: A lightweight 3D CNN with global topological features for preterm brain injury predictioncited by 1
25 more
  1. 2026WaveDiT: Distribution-Aware Wavelet Flow Matching for Efficient 3D Brain MRI Synthesiscited by 1
  2. 2026BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generationcited by 1
  3. 2026Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRIcited by 1
  4. 2025TAlignDiff: Automatic Tooth Alignment assisted by Diffusion-based Transformation Learningcited by 0
  5. 2025Spatio-Temporal Conditional Diffusion Models for Forecasting Future Multiple Sclerosis Lesion Masks Conditioned on Treatmentscited by 0
  6. 2025K-Syn: K-space Data Synthesis in Ultra Low-data Regimescited by 0
  7. 2025ALL-PET: A Low-resource and Low-shot PET Foundation Model in Projection Domaincited by 0
  8. 2025BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNetcited by 0
  9. 2026Joint 3D Gravity and Magnetic Inversion via Rectified Flow and Ginzburg-Landau Guidancecited by 0
  10. 2026VolDiT: Controllable Volumetric Medical Image Synthesis with Diffusion Transformerscited by 0
  11. 2026CardioDiT: Latent Diffusion Transformers for 4D Cardiac MRI Synthesiscited by 0
  12. 2026PHASOR: Anatomy- and Phase-Consistent Volumetric Diffusion for CT Virtual Contrast Enhancementcited by 0
  13. 2026Few-Shot Distribution-Aligned Flow Matching for Data Synthesis in Medical Image Segmentationcited by 0
  14. 2026Distilling Photon-Counting CT into Routine Chest CT through Clinically Validated Degradation Modelingcited by 0
  15. 2026Weakly-Supervised Lung Nodule Segmentation via Training-Free Guidance of 3D Rectified Flowcited by 0
  16. 2026LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generatorscited by 0
  17. 2026MedSyn2: Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Promptscited by 0
  18. 2026Multiscale Hypersonic Boundary Layer Reconstruction via Spectral Binning and Subdomain-wise Conditional Diffusioncited by 0
  19. 2026TensorLDM: A Component-Wise Latent Diffusion Model for Volumetric DTI Reconstruction from Sparse DWIscited by 0
  20. 2026X-Splat: Gaussian Splatting for 3D CBCT Generation from Single Panoramic Radiographcited by 0
  21. 2026CONFLUX: A Latent Diffusion Model for 3D Chest-CT Synthesis with RL Post-Trainingcited by 0
  22. 2026VoxStruct3D: Structure-Leading Flow Matching for Voxel-Space 3D MRI Synthesiscited by 0
  23. 2026THA-Flow Generative Model: Prosthesis Geometry Prediction from Preoperative CTcited by 0
  24. 2026AWR-Net: Decoupling Anatomy and Appearance for 3D Fetal Brain Ultrasound Synthesiscited by 0
  25. 2026What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processing
  2. 2024MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications
  3. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Review
  4. 2026Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation
  5. 2024Similarity-aware Syncretic Latent Diffusion Model for Medical Image Translation with Representation Learning
  6. 2025Local Patches Meet Global Context: Scalable 3D Diffusion Priors for Computed Tomography Reconstruction
  7. 2024DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction
  8. 2026Structure-Adaptive Sparse Diffusion in Voxel Space for 3D Medical Image Enhancement