mlsanityData to 7 Oct 2026

Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation

arXiv 2510.06131 ↗First posted 7 Oct 2025cs.CV cs.AIcited by 1

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Recent advances in generative medical models are constrained by modality-specific scenarios that hinder the integration of complementary evidence from imaging, pathology, and clinical notes. This fragmentation limits their evolution into foundation models that can learn and reason across the full spectrum of biomedical data. We propose MeDiM, the first medical discrete diffusion model that learns shared distributions across modalities without modality-specific components. MeDiM unifies multiple generative tasks: translating between images and text, and jointly producing image-report pairs across domains in response to prompts. Built on a discrete diffusion framework, MeDiM bridges vision and language representations through a shared probabilistic space. To enable unified and flexible medical generation, we employ a multimodal large language model (MLLM) as the diffusion backbone, leveraging its prior knowledge and cross-modal reasoning. Two key designs are introduced: (1) removing the causal attention mask for bidirectional context, and (2) injecting continuous timestep embeddings for diffusion awareness. Experiments demonstrate high-fidelity medical generation (FID 16.60 on MIMIC-CXR and FID 24.19 on PathGen) and accurate report generation (METEOR 0.2650 and 0.2580). Jointly generated image-report pairs further enhance downstream performance (plus6.43 percent BLEU-1, plus18.57 percent BLEU-2, plus31.58 percent BLEU-3, plus4.80 percent METEOR), showing that MeDiM supports coherent and clinically grounded multimodal outputs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 55 of its 65 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

55 of 65 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Med-Art: Diffusion Transformer for 2D Medical Text-to-Image Generation25 Jun
  2. 2024Leapfrog Latent Consistency Model (LLCM) for Medical Images Generation22 Nov
  3. 2024Fast-DDPM: Fast Denoising Diffusion Probabilistic Models for Medical Image-to-Image Generation23 May
  4. 2024MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant7 Mar
  5. 2023DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model19 Oct
  6. 2022Adaptive Diffusion Priors for Accelerated MRI Reconstruction12 Jul

Other references · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2017Neural Discrete Representation Learningcited by 4,650
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
37 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  7. 2025Large Language Diffusion Modelscited by 872
  8. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  11. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  12. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  13. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  14. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  15. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  16. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  17. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  18. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  19. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  20. 2024U-KAN Makes Strong Backbone for Medical Image Segmentation and Generationcited by 176
  21. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
  22. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  23. 2023Planting a SEED of Vision in Large Language Modelcited by 140
  24. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  25. 2022RoentGen: Vision-Language Foundation Model for Chest X-ray Generationcited by 89
  26. 2022Conversion Between CT and MRI Images Using Diffusion and Score-Matching Modelscited by 69
  27. 2018SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truthcited by 68
  28. 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
  29. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
  30. 2025Unified Multimodal Discrete Diffusioncited by 46
  31. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  32. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 39
  33. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  34. 2023Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generationcited by 25
  35. 2025MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairscited by 13
  36. 2024PairAug: What Can Augmented Image-Text Pairs Do for Radiology?cited by 10
  37. 2024Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2022Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains
  2. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine
  3. 2026MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images
  4. 2024Dual Diffusion for Unified Image Generation and Understanding
  5. 2025Language-Guided Trajectory Traversal in Disentangled Stable Diffusion Latent Space for Factorized Medical Image Generation
  6. 2025LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding
  7. 2025VAP-Diffusion: Enriching Descriptions with MLLMs for Enhanced Medical Image Generation
  8. 2024Mediffusion: Joint Diffusion for Self-Explainable Semi-Supervised Classification and Medical Image Generation