mlsanityData to 7 Oct 2026

ResViT: Residual vision transformers for multi-modal medical image synthesis

arXiv 2106.16031 ↗First posted 30 Jun 2021eess.IV cs.CVcited by 177

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative adversarial models with convolutional neural network (CNN) backbones have recently been established as state-of-the-art in numerous medical image synthesis tasks. However, CNNs are designed to perform local processing with compact filters, and this inductive bias compromises learning of contextual features. Here, we propose a novel generative adversarial approach for medical image synthesis, ResViT, that leverages the contextual sensitivity of vision transformers along with the precision of convolution operators and realism of adversarial learning.} ResViT's generator employs a central bottleneck comprising novel aggregated residual transformer (ART) blocks that synergistically combine residual convolutional and transformer modules. Residual connections in ART blocks promote diversity in captured representations, while a channel compression module distills task-relevant information. A weight sharing strategy is introduced among ART blocks to mitigate computational burden. A unified implementation is introduced to avoid the need to rebuild separate synthesis models for varying source-target modality configurations. Comprehensive demonstrations are performed for synthesizing missing sequences in multi-contrast MRI, and CT images from MRI. Our results indicate superiority of ResViT against competing CNN- and transformer-based methods in terms of qualitative observations and quantitative metrics.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Builds on 51 of its 119 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

51 of 119 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2017Non-local Neural Networkscited by 2,290
  8. 2014Conditional Generative Adversarial Netscited by 2,152
  9. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  11. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,420
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
38 more
  1. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  2. 2018Self-Attention Generative Adversarial Networkscited by 809
  3. 2020Quantifying Attention Flow in Transformerscited by 759
  4. 2021ImageNet-21K Pretraining for the Massescited by 607
  5. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  6. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 278
  7. 2018Attention Gated Networks: Learning to Leverage Salient Regions in Medical Imagescited by 271
  8. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  9. 2018GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classificationcited by 242
  10. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  11. 2018MedGAN: Medical Image Translation using GANscited by 152
  12. 2018Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networkscited by 131
  13. 2021An Attention Free Transformercited by 111
  14. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  15. 2017Deep MR to CT Synthesis using Unpaired Datacited by 86
  16. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  17. 2021TransMed: Transformers Advance Multi-modal Medical Image Classificationcited by 85
  18. 2021Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformerscited by 76
  19. 2021Convolution-Free Medical Image Segmentation using Transformerscited by 49
  20. 2018Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data sizecited by 42
  21. 2019DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesiscited by 36
  22. 2021TransCT: Dual-path Transformer for Low Dose Computed Tomographycited by 35
  23. 2019mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesiscited by 34
  24. 2017Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Codingcited by 33
  25. 2019CollaGAN : Collaborative GAN for Missing Image Data Imputationcited by 33
  26. 2018Deep CT to MR Synthesis using Paired and Unpaired Datacited by 31
  27. 2018Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGANcited by 31
  28. 2017Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truthcited by 30
  29. 2018Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detectioncited by 29
  30. 2021VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformerscited by 28
  31. 2017The Devil is in the Decoder: Classification, Regression and GANscited by 22
  32. 2018High-resolution medical image synthesis using progressively grown generative adversarial networkscited by 17
  33. 2020GANBERT: Generative Adversarial Networks with Bidirectional Encoder Representations from Transformers for MRI to PET synthesiscited by 15
  34. 2020Progressively Volumetrized Deep Generative Models for Data-Efficient Contextual Learning of MR Image Recoverycited by 14
  35. 2021PTNet: A High-Resolution Infant MRI Synthesizer Based on Transformercited by 11
  36. 2019Improved MR to CT synthesis for PET/MR attenuation correction using Imitation Learningcited by 3
  37. 2020Semi-Supervised Learning of Mutually Accelerated MRI Synthesis without Fully-Sampled Ground Truthscited by 3
  38. 2021Convolutional Block Design for Learned Fractional Downsamplingcited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 26; 2023: 30; 2024: 48; 2025: 37; 2026: 33.

177 papers.

Sort:

By area:

  1. 2022Transformers in Medical Imaging: A Surveycited by 245
  2. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Modelscited by 191
  3. 2022Adaptive Diffusion Priors for Accelerated MRI Reconstructioncited by 117
  4. 2023CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesiscited by 64
  5. 2023Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Studycited by 64
  6. 2023Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Reviewcited by 62
  7. 2022Swin Transformer for Fast MRIcited by 56
  8. 2022ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformercited by 52
  9. 2023Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesiscited by 43
  10. 2022Transformers in Medical Image Analysis: A Reviewcited by 41
  11. 2022One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputationcited by 40
  12. 2023Unified Multi-Modal Image Synthesis for Missing Modality Imputationcited by 39
38 more
  1. 2021Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative modelscited by 36
  2. 2022Class-Aware Adversarial Transformers for Medical Image Segmentationcited by 33
  3. 2022Federated Learning of Generative Image Priors for MRI Reconstructioncited by 33
  4. 2024Self-Consistent Recursive Diffusion Bridge for Medical Image Translationcited by 26
  5. 2024I2I-Mamba: Multi-modal medical image synthesis via selective state space modelingcited by 26
  6. 2024Cross-conditioned Diffusion Model for Medical Image to Image Translationcited by 26
  7. 2023Medical Vision Language Pretraining: A surveycited by 24
  8. 2022BolT: Fused Window Transformers for fMRI Time Series Analysiscited by 23
  9. 2022One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesiscited by 22
  10. 2023Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translationcited by 19
  11. 2022Medical Image Segmentation on MRI Images with Missing Modalities: A Reviewcited by 18
  12. 2024StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translationcited by 18
  13. 2023Transformers in Healthcare: A Surveycited by 17
  14. 2021Data synthesis and adversarial networks: A review and meta-analysis in cancer imagingcited by 15
  15. 2024Derm-T2IM: Harnessing Synthetic Skin Lesion Data via Stable Diffusion Models for Enhanced Skin Disease Classification using ViT and CNNcited by 15
  16. 2024FairDiff: Fair Segmentation with Point-Image Diffusioncited by 15
  17. 2023DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRIcited by 14
  18. 2022Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesiscited by 13
  19. 2023Dual-Domain Self-Supervised Learning for Accelerated Non-Cartesian MRI Reconstructioncited by 13
  20. 2023Learning Fourier-Constrained Diffusion Bridges for MRI Reconstructioncited by 13
  21. 2023Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representationscited by 13
  22. 2023MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translationcited by 13
  23. 20242.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation: Application to Low-count PET Reconstruction with CT-less Attenuation Correctioncited by 13
  24. 2025MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairscited by 13
  25. 2023LIT-Former: Linking In-plane and Through-plane Transformers for Simultaneous CT Image Denoising and Deblurringcited by 12
  26. 2023Is attention all you need in medical image analysis? A reviewcited by 12
  27. 2025UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysiscited by 12
  28. 2023Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRIcited by 11
  29. 2024DenoMamba: A fused state-space model for low-dose CT denoisingcited by 11
  30. 2023Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Surveycited by 10
  31. 2024PairAug: What Can Augmented Image-Text Pairs Do for Radiology?cited by 10
  32. 2024PASTA: Pathology-Aware MRI to PET Cross-Modal Translation with Diffusion Modelscited by 10
  33. 2024Multimodal Data Integration for Precision Oncology: Challenges and Future Directionscited by 10
  34. 2022Recent Progress in Transformer-based Medical Image Analysiscited by 9
  35. 2023An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesiscited by 9
  36. 2024Ultrasound Nodule Segmentation Using Asymmetric Learning with Simple Clinical Annotationcited by 9
  37. 2024VM-DDPM: Vision Mamba Diffusion for Medical Image Synthesiscited by 8
  38. 2024Structural Attention: Rethinking Transformer for Unpaired Medical Image Synthesiscited by 8

50 most cited of 177 All 177 (JSON)