# MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant arXiv: https://arxiv.org/abs/2403.04290 ยท HTML: https://mlsanity.com/paper/2403.04290/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. - Builds on 17 of its 63 references that are in the map. - First posted on arXiv 7 Mar 2024. ## Placed in - Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/) ## Builds on (most cited first, 17 of 17) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018) - [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017) - [Any-to-Any Generation via Composable Diffusion](https://mlsanity.com/paper/2305.11846/) (2023) - [UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.05358/) (2021) - [Noise Estimation for Generative Diffusion Models](https://mlsanity.com/paper/2104.02600/) (2021) - [RoentGen: Vision-Language Foundation Model for Chest X-ray Generation](https://mlsanity.com/paper/2211.12737/) (2022) - [MedICaT: A Dataset of Medical Images, Captions, and Textual References](https://mlsanity.com/paper/2010.06000/) (2020) - [Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models](https://mlsanity.com/paper/2209.12104/) (2022) - [Complex Organ Mask Guided Radiology Report Generation](https://mlsanity.com/paper/2311.02329/) (2023) - [Vision-Language Generative Model for View-Specific Chest X-ray Generation](https://mlsanity.com/paper/2302.12172/) (2023) - [Robust Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1805.08657/) (2018) ## Built on by (most cited first, 21 of 21) - [Sora as a World Model? A Complete Survey on Text-to-Video Generation](https://mlsanity.com/paper/2403.05131/) (2024) - [Libra: Leveraging Temporal Images for Biomedical Radiology Analysis](https://mlsanity.com/paper/2411.19378/) (2024) - [MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations](https://mlsanity.com/paper/2503.01019/) (2025) - [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024) - [Longitudinal Flow Matching for Trajectory Modeling](https://mlsanity.com/paper/2510.03569/) (2025) - [Physics-Inspired Generative Models in Medical Imaging: A Review](https://mlsanity.com/paper/2407.10856/) (2024) - [DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching](https://mlsanity.com/paper/2412.14535/) (2024) - [Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation](https://mlsanity.com/paper/2411.06106/) (2024) - [MRGen: Segmentation Data Engine for Underrepresented MRI Modalities](https://mlsanity.com/paper/2412.04106/) (2024) - [MMLNB: Multi-Modal Learning for Neuroblastoma Subtyping Classification Assisted with Textual Description Generation](https://mlsanity.com/paper/2503.12927/) (2025) - [Scalable next-scale autoregression for medical image generation across anatomical regions](https://mlsanity.com/paper/2602.14512/) (2026) - [I2I-Galip: Unsupervised Medical Image Translation Using Generative Adversarial CLIP](https://mlsanity.com/paper/2409.12399/) (2024) - [Towards Better Cephalometric Landmark Detection with Diffusion Data Generation](https://mlsanity.com/paper/2505.06055/) (2025) - [Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge](https://mlsanity.com/paper/2602.16664/) (2026) - [MExD: An Expert-Infused Diffusion Model for Whole-Slide Image Classification](https://mlsanity.com/paper/2503.12401/) (2025) - [Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation](https://mlsanity.com/paper/2510.06131/) (2025) - [ReBrain: Brain MRI Reconstruction from Sparse CT Slice via Retrieval-Augmented Diffusion](https://mlsanity.com/paper/2511.17068/) (2025) - [Any-to-All MRI Synthesis: A Unified Foundation Model for Nasopharyngeal Carcinoma and Its Downstream Applications](https://mlsanity.com/paper/2602.08822/) (2026) - [Unified MRI Brain Image Translation via Hierarchical Tumor Structure Comparison](https://mlsanity.com/paper/2606.13096/) (2026) - [Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling](https://mlsanity.com/paper/2607.09892/) (2026) - [NuclearDiffusion: Text-to-Image Foundation Models for Learning Nuclear Energy Concepts](https://mlsanity.com/paper/2608.04030/) (2026) ## Abstract (arXiv) Medical generative models, acknowledged for their high-quality sample generation ability, have accelerated the fast growth of medical applications. However, recent works concentrate on separate medical generation models for distinct medical tasks and are restricted to inadequate medical multi-modal knowledge, constraining medical comprehensive diagnosis. In this paper, we propose MedM2G, a Medical Multi-Modal Generative framework, with the key innovation to align, extract, and generate medical multi-modal within a unified model. Extending beyond single or two medical modalities, we efficiently align medical multi-modal through the central alignment approach in the unified space. Significantly, our framework extracts valuable clinical knowledge by preserving the medical visual invariant of each imaging modal, thereby enhancing specific medical information for multi-modal generation. By conditioning the adaptive cross-guided parameters into the multi-flow diffusion framework, our model promotes flexible interactions among medical multi-modal for generation. MedM2G is the first medical generative model that unifies medical generation tasks of text-to-image, image-to-text, and unified generation of medical modalities (CT, MRI, X-ray). It performs 5 medical generation tasks across 10 datasets, consistently outperforming various state-of-the-art works.