mlsanityData to 7 Oct 2026

Spatio-Temporal Mixture-of-Modality-Experts Diffusion for Quantitative DCE-MRI Synthesis from Incomplete MR Sequences

arXiv 2606.25535 ↗First posted 24 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantitative maps from dynamic contrast-enhanced MRI (DCE-MRI) are essential for tumor assessment but are often unavailable due to contrast-agent risks and protocol variability. Prior methods predict these maps from other MRI modalities, yet most assume fixed, fully observed inputs and fail under realistic missingness. We present Spatio-Temporal Mixture-of-Modality-Experts (ST-MoME), a conditional diffusion framework that synthesizes 3D DCE parameter maps from diverse subsets of multimodal MRI. ST-MoME fuses modality-specific expert features through a spatio-temporal gating network that produces voxel-wise, timestep-dependent weights, forming a conditioning tensor that guides denoising. To preserve quantitative fidelity, ST-MoME performs diffusion directly in image space with 3D patch-based training and a Swin-based backbone. On a clinical brain-tumor cohort of 386 patients, we evaluate ST-MoME across 16 controlled modality-availability scenarios. It achieves the lowest mean Normalized Mean Square Error (NMSE) aggregated across all three DCE parameters, with leading performance on vp and ve, competitive results on Ktrans, and the lowest reconstruction error within the clinically critical tumor region. A post-hoc analysis of the learned gating dynamics shows a structural-early, physiological-late fusion schedule consistent with clinical intuition.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

25 of 41 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  9. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  10. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 197
  11. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  12. 2016HeMIS: Hetero-Modal Image Segmentationcited by 102
13 more
  1. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  2. 2024MAISI: Medical AI for Synthetic Imagingcited by 86
  3. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
  4. 2023CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesiscited by 64
  5. 2023Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Studycited by 64
  6. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  7. 2024MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Modelscited by 45
  8. 2023Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processingcited by 30
  9. 2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 22
  10. 2024Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problemscited by 18
  11. 2024PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Ratescited by 17
  12. 2021Glioblastoma Multiforme Prognosis: MRI Missing Modality Generation, Segmentation and Radiogenomic Survival Predictioncited by 5
  13. 2025From 2D to 3D, Deep Learning-based Shape Reconstruction in Magnetic Resonance Imaging: A Reviewcited by 2

Built on by

Not cited yet.