mlsanityData to 7 Oct 2026

A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Models

arXiv 2602.08249 ↗First posted 9 Feb 2026eess.IV cs.CVcited by 1

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Image reconstruction and image synthesis are important for handling incomplete multimodal imaging data, but existing methods require various task-specific models, complicating training and deployment workflows. We introduce Any2all, a unified framework that addresses this limitation by formulating these disparate tasks as a single virtual inpainting problem. We train a single, unconditional diffusion model on the complete multimodal data stack. This model is then adapted at inference time to ``inpaint'' all target modalities from any combination of inputs of available clean images or noisy measurements. We validated Any2all on a PET/MR/CT brain dataset. Our results show that Any2all can achieve excellent performance on both multimodal reconstruction and synthesis tasks, consistently yielding images with competitive distortion-based performance and superior perceptual quality over specialized methods.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 27 of its 40 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

27 of 40 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution7 Apr
  2. 2023Synthetic CT Generation from MRI using 3D Transformer-based Denoising Diffusion Model31 May
  3. 2023MRI Reconstruction with Side Information using Diffusion Models26 Mar
  4. 2023CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis24 Mar
  5. 2023DisC-Diff: Disentangled Conditional Diffusion Model for Multi-Contrast MRI Super-Resolution24 Mar
  6. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Models17 Jul

Other references · 21

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  5. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  6. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  7. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
  8. 2017Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)cited by 224
  9. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  10. 2015Brain Tumor Segmentation with Deep Neural Networkscited by 137
  11. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 129
  12. 2019Multimodal End-to-End Autonomous Drivingcited by 110
9 more
  1. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  2. 2022Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imagingcited by 94
  3. 2017A Review of Convolutional Neural Networks for Inverse Problems in Imagingcited by 81
  4. 2020DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Priorcited by 72
  5. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  6. 2021Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Networkcited by 42
  7. 2022One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputationcited by 40
  8. 2021Deformation-Compensated Learning for Image Reconstruction without Ground Truthcited by 9
  9. 2023Block Coordinate Plug-and-Play Methods for Blind Inverse Problemscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Information
  2. 2023Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
  3. 2023Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction
  4. 2026Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions
  5. 2026Adaptive Clinical-Aware Latent Diffusion for Multimodal Brain Image Generation and Missing Modality Imputation
  6. 2025MAISI-v2: Accelerated 3D High-Resolution Medical Image Synthesis with Rectified Flow and Region-specific Contrastive Loss
  7. 2023Compensation Sampling for Improved Convergence in Diffusion Models
  8. 2023Residual Denoising Diffusion Models