mlsanityData to 7 Oct 2026

Unified Multi-Modal Image Synthesis for Missing Modality Imputation

arXiv 2304.05340 ↗First posted 11 Apr 2023cs.CV eess.IVcited by 39

Not on the map. Why?

Abstract (arXiv)

Multi-modal medical images provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-modal images, thus limiting the usage of multi-modal data for clinical purposes. To address this issue, in this paper, we propose a novel unified multi-modal image synthesis method for missing modality imputation. Our method overall takes a generative adversarial architecture, which aims to synthesize missing modalities from any combination of available ones with a single model. To this end, we specifically design a Commonality- and Discrepancy-Sensitive Encoder for the generator to exploit both modality-invariant and specific information contained in input modalities. The incorporation of both types of information facilitates the generation of images with consistent anatomy and realistic details of the desired distribution. Besides, we propose a Dynamic Feature Unification Module to integrate information from a varying number of available modalities, which enables the network to be robust to random missing modalities. The module performs both hard integration and soft integration, ensuring the effectiveness of feature combination while avoiding information loss. Verified on two public multi-modal magnetic resonance datasets, the proposed method is effective in handling various synthesis tasks and shows superior performance compared to previous methods.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 27 of its 61 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

27 of 61 references.

References · 27

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2020Knowledge Distillation: A Surveycited by 1,578
  4. 2016Least Squares Generative Adversarial Networkscited by 1,000
  5. 2020A Survey on Visual Transformercited by 653
  6. 2018Rethinking the Value of Network Pruningcited by 595
  7. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  8. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Modelscited by 191
  9. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  10. 2018Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networkscited by 131
  11. 2016HeMIS: Hetero-Modal Image Segmentationcited by 102
  12. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
15 more
  1. 2016Medical Image Synthesis with Context-Aware Generative Adversarial Networkscited by 91
  2. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  3. 2019Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentationcited by 68
  4. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
  5. 2021Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalitiescited by 42
  6. 2022One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputationcited by 40
  7. 2019DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesiscited by 36
  8. 2019mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesiscited by 34
  9. 2019CollaGAN : Collaborative GAN for Missing Image Data Imputationcited by 33
  10. 2020Multi-Domain Image Completion for Random Missing Input Datacited by 30
  11. 2022One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesiscited by 22
  12. 2019A unified representation network for segmentation with missing modalitiescited by 15
  13. 2018Generative Adversarial Training for MRA Image Synthesis Using Multi-Contrast MRIcited by 10
  14. 2022Undersampled MRI Reconstruction with Side Information-Guided Normalisationcited by 5
  15. 2022LungViT: Ensembling Cascade of Texture Sensitive Hierarchical Vision Transformers for Cross-Volume Chest CT Image-to-Image Translationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 13; 2026: 14.

39 papers.

Sort:

By area:

  1. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
  2. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 58
  3. 2024Self-Consistent Recursive Diffusion Bridge for Medical Image Translationcited by 26
  4. 2024MMP: Towards Robust Multi-Modal Learning with Masked Modality Projectioncited by 10
  5. 2023Multi-modal Latent Diffusioncited by 9
  6. 2025Regression is all you need for medical image translationcited by 8
  7. 2024Learning Modality-Aware Representations: Adaptive Group-wise Interaction Network for Multimodal MRI Synthesiscited by 4
  8. 2025Virtual Full-stack Scanning of Brain MRI via Imputing Any Quantised Codecited by 4
  9. 2024Federated Learning for Time-Series Healthcare Sensing with Incomplete Modalitiescited by 3
  10. 2025FgC2F-UDiff: Frequency-guided and Coarse-to-fine Unified Diffusion Model for Multi-modality Missing MRI Synthesiscited by 3
  11. 2025Augmented Intelligence for Multimodal Virtual Biopsy in Breast Cancer Using Generative Artificial Intelligencecited by 3
  12. 2025Interactive Gadolinium-Free MRI Synthesis: A Transformer with Localization Prompt Learningcited by 3
27 more
  1. 2025SAGCNet: Spatial-Aware Graph Completion Network for Missing Slice Imputation in Population CMR Imagingcited by 3
  2. 2024Multi-modal MRI Translation via Evidential Regression and Distribution Calibrationcited by 2
  3. 2024DuDoUniNeXt: Dual-domain unified hybrid model for single and multi-contrast undersampled MRI reconstructioncited by 1
  4. 2025Deep infant brain segmentation from multi-contrast MRIcited by 1
  5. 2026ModalImmune: Immunity Driven Unlearning via Self Destructive Trainingcited by 1
  6. 2026Multimodal synthesis of MRI and tabular data with diffusion in a joint latent space via cross-attentioncited by 1
  7. 2024HyperMM : Robust Multimodal Learning with Varying-sized Inputscited by 0
  8. 2024Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalitiescited by 0
  9. 2024Longitudinal Causal Image Synthesiscited by 0
  10. 2025Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosiscited by 0
  11. 2025BiXFormer: A Robust Framework for Maximizing Modality Effectiveness in Multi-Modal Semantic Segmentationcited by 0
  12. 2025DMAF-Net: An Effective Modality Rebalancing Framework for Incomplete Multi-Modal Medical Image Segmentationcited by 0
  13. 2025F3-Net: Foundation Model for Full Abnormality Segmentation of Medical Images with Flexible Input Modality Requirementcited by 0
  14. 2025Pyramid Hierarchical Masked Diffusion Model for Imaging Synthesiscited by 0
  15. 2025CLARGA: Multimodal Graph Representation Learning over Arbitrary Sets of Modalitiescited by 0
  16. 2026OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflowscited by 0
  17. 2026Any-to-All MRI Synthesis: A Unified Foundation Model for Nasopharyngeal Carcinoma and Its Downstream Applicationscited by 0
  18. 2026Towards Personalized Multi-Modal MRI Synthesis across Heterogeneous Datasetscited by 0
  19. 2026DeLo: Dual Decomposed Low-Rank Experts Collaboration for Continual Missing Modality Learningcited by 0
  20. 2026Triple-Phase Sequential Fusion Network for Hepatobiliary Phase Liver MRI Synthesiscited by 0
  21. 2026Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Datacited by 0
  22. 2026Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classificationcited by 0
  23. 2026Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentationcited by 0
  24. 2026A cross-modal generative model for incomplete and degraded prostate MRI with multicentre clinical validationcited by 0
  25. 2026Residual Optimal Transport-Based Experts Collaboration Towards Modality-Aware Infrared-Visible Object Detectioncited by 0
  26. 2026VIS-DICT: A Visual Dictionary for Missing Modality Imputation in Social Network Depression Detectioncited by 0
  27. 2026PredRA: Fast Medical Image Translation by Deterministic Component Extraction and Controlled Stochastic Refinementcited by 0