mlsanityData to 7 Oct 2026

MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications

arXiv 2410.15432 ↗First posted 20 Oct 2024cs.CVcited by 7

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion models have achieved significant success in both natural image and medical image domains, encompassing a wide range of applications. Previous investigations in medical images have often been constrained to specific anatomical regions, particular applications, and limited datasets, resulting in isolated diffusion models. This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM. MedDiff-FM leverages 3D CT images from multiple publicly available datasets, covering anatomical regions from head to abdomen, to pre-train a diffusion foundation model, and explores the capabilities of the diffusion foundation model across a variety of application scenarios. The diffusion foundation model handles multi-level integrated image processing both at the image-level and patch-level, utilizes position embedding to establish multi-level spatial relationships, and leverages region classes and anatomical structures to capture certain anatomical regions. MedDiff-FM manages several downstream tasks seamlessly, including image denoising, anomaly detection, and image synthesis. MedDiff-FM is also capable of performing super-resolution, lesion generation, and lesion inpainting by rapidly fine-tuning the diffusion foundation model using ControlNet with task-specific conditions. The experimental results demonstrate the effectiveness of MedDiff-FM in addressing diverse downstream medical image tasks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 71 of its 99 references that are in the map. First posted on arXiv 20 Oct 2024.

Builds on

71 of 99 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024MAISI: Medical AI for Synthetic Imaging13 Sep
  2. 2024DiNO-Diffusion. Scaling Medical Diffusion via Self-Supervised Pre-Training16 Jul
  3. 2023Semantic Image Synthesis for Abdominal CT11 Dec
  4. 2023DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model19 Oct
  5. 2023ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models3 Sep
  6. 2023Mask, Stitch, and Re-Sample: Enhancing Robustness and Generalizability in Anomaly Detection through Automatic Diffusion Models31 May
  7. 2023Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis29 May
  8. 2023Diffusion Probabilistic Priors for Zero-Shot Low-Dose CT Image Denoising25 May
  9. 2023CoreDiff: Contextual Error-Modulated Generalized Diffusion Model for Low-Dose CT Denoising and Generalization4 Apr
  10. 2023NASDM: Nuclei-Aware Semantic Histopathology Image Generation Using Diffusion Models20 Mar
  11. 2023DDM2: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Models6 Feb
  12. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Survey14 Nov
2 more
  1. 2022Brain Imaging Generation with Latent Diffusion Models15 Sep
  2. 2022Diffusion Models for Medical Anomaly Detection8 Mar

Other references · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2023Consistency Modelscited by 1,778
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
38 more
  1. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  2. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  3. 2022Diffusion Models in Vision: A Surveycited by 1,008
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  5. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  6. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  7. 2021The Medical Segmentation Decathloncited by 620
  8. 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
  9. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 591
  10. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 510
  11. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 419
  12. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 371
  13. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shiftingcited by 360
  14. 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
  15. 2023Denoising Diffusion Models for Plug-and-Play Image Restorationcited by 335
  16. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  17. 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 307
  18. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  19. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 282
  20. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  21. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  22. 2023SimpleNet: A Simple Network for Image Anomaly Detection and Localizationcited by 256
  23. 2017Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)cited by 224
  24. 2020AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?cited by 222
  25. 2022Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Modelscited by 215
  26. 2022CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localizationcited by 135
  27. 2022Semantic Image Synthesis via Diffusion Modelscited by 131
  28. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 131
  29. 2023BMAD: Benchmarks for Medical Anomaly Detectioncited by 81
  30. 2024Towards Generalizable Tumor Synthesiscited by 65
  31. 2014A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observationscited by 56
  32. 2018Can Deep Learning Outperform Modern Commercial CT Image Reconstruction Methods?cited by 53
  33. 2022CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoisingcited by 46
  34. 2023Text-image Alignment for Diffusion-based Perceptioncited by 42
  35. 2023MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Imagescited by 41
  36. 2020SAINT: Spatially Aware Interpolation NeTwork for Medical Slice Synthesiscited by 37
  37. 2022High-Fidelity Guided Image Synthesis with Latent Diffusion Modelscited by 36
  38. 2023MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generationcited by 34

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translationcited by 8
  2. 2026Scalable next-scale autoregression for medical image generation across anatomical regionscited by 3
  3. 2025A-IDE : Agent-Integrated Denoising Expertscited by 1
  4. 2026BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generationcited by 1
  5. 2025SkinDualGen: Prompt-Driven Diffusion for Simultaneous Image-Mask Generation in Skin Lesionscited by 0
  6. 2025Establishing Stochastic Object Models from Noisy Data via Ambient Measurement-Integrated Diffusioncited by 0
  7. 2026Multiscale Hypersonic Boundary Layer Reconstruction via Spectral Binning and Subdomain-wise Conditional Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions
  2. 20243D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-quality Medical Image Generation
  3. 2025MedDIFT: Multi-Scale Diffusion-Based Correspondence in 3D Medical Imaging
  4. 2025MedCondDiff: Lightweight, Robust, Semantically Guided Diffusion for Medical Image Segmentation
  5. 2023FDDM: Unsupervised Medical Image Translation with a Frequency-Decoupled Diffusion Model
  6. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Review
  7. 2023Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processing
  8. 2025MedDiff-FT: Data-Efficient Diffusion Model Fine-tuning with Structural Guidance for Controllable Medical Image Synthesis