mlsanityData to 7 Oct 2026

Cross-Modality Neuroimage Synthesis: A Survey

arXiv 2202.06997 ↗First posted 14 Feb 2022eess.IV cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. However, collecting fully aligned and paired data is expensive or even impractical, since it faces many difficulties, including high cost, long acquisition time, image corruption, and privacy issues. An alternative solution is to explore unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data. In this paper, we provide a comprehensive review of cross-modality synthesis for neuroimages, from the perspectives of weakly supervised and unsupervised settings, loss functions, evaluation metrics, imaging modalities, datasets, and downstream applications based on synthesis. We begin by highlighting several opening challenges for cross-modality neuroimage synthesis. Then, we discuss representative architectures of cross-modality synthesis methods under different supervisions. This is followed by a stepwise in-depth analysis to evaluate how cross-modality neuroimage synthesis improves the performance of its downstream tasks. Finally, we summarize the existing research findings and point out future research directions. All resources are available at https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 42 of its 148 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

42 of 148 references.

References · 42

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  4. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  5. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 614
  6. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  7. 2016Texture Networks: Feed-forward Synthesis of Textures and Stylized Imagescited by 173
  8. 2020Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentationcited by 134
  9. 2018Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networkscited by 131
  10. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  11. 2018Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Networkcited by 97
  12. 2016Medical Image Synthesis with Context-Aware Generative Adversarial Networkscited by 91
30 more
  1. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  2. 2021Breaking the Dilemma of Medical Image-to-image Translationcited by 82
  3. 2018SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truthcited by 68
  4. 2019Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networkscited by 65
  5. 2020Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translationcited by 60
  6. 2014Sparse Modeling for Image and Vision Processingcited by 48
  7. 2018Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data sizecited by 42
  8. 2019Learning Fixed Points in Generative Adversarial Networks: From Image-to-Image Translation to Disease Detection and Localizationcited by 38
  9. 2020Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesiscited by 38
  10. 2019DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesiscited by 36
  11. 2019mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesiscited by 34
  12. 2017Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Codingcited by 33
  13. 2019CollaGAN : Collaborative GAN for Missing Image Data Imputationcited by 33
  14. 2018An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detectioncited by 30
  15. 2020Multi-Domain Image Completion for Random Missing Input Datacited by 30
  16. 2021Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptationcited by 24
  17. 2018Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentationcited by 21
  18. 2022FedMed-GAN: Federated Domain Translation on Unsupervised Cross-Modality Brain Image Synthesiscited by 21
  19. 2021A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translationcited by 19
  20. 2020Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesiscited by 18
  21. 2021A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translationcited by 15
  22. 2020GANDALF: Generative Adversarial Networks with Discriminator-Adaptive Loss Fine-tuning for Alzheimer's Disease Diagnosis from MRIcited by 13
  23. 2020Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesiscited by 12
  24. 2018Generative Adversarial Training for MRA Image Synthesis Using Multi-Contrast MRIcited by 10
  25. 2013Sparsity Based Poisson Denoising with Dictionary Learningcited by 9
  26. 2021Anatomy-Constrained Contrastive Learning for Synthetic Segmentation without Ground-truthcited by 9
  27. 2018Learning Myelin Content in Multiple Sclerosis from Multimodal MRI through Adversarial Trainingcited by 7
  28. 2021Synthesizing Multi-Tracer PET Images for Alzheimer's Disease Patients using a 3D Unified Anatomy-aware Cyclic Adversarial Networkcited by 5
  29. 2017DOTE: Dual cOnvolutional filTer lEarning for Super-Resolution and Cross-Modality Synthesis in MRIcited by 3
  30. 2022FedMed-ATL: Misaligned Unpaired Brain Image Synthesis via Affine Transform Losscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4.

6 papers.

  1. 2023Domain Generalization for Medical Image Analysis: A Reviewcited by 47
  2. 2024A review of deep learning-based information fusion techniques for multimodal medical image classificationcited by 25
  3. 2024Medical Vision Generalist: Unifying Medical Imaging Tasks in Contextcited by 13
  4. 2024Navigating Distribution Shifts in Medical Image Analysis: A Surveycited by 5
  5. 2024Cross-modality image synthesis from TOF-MRA to CTA using diffusion-based modelscited by 4
  6. 2023fRegGAN with K-space Loss Regularization for Medical Image Translationcited by 3