mlsanityData to 7 Oct 2026

Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review

arXiv 2301.03505 ↗First posted 9 Jan 2023cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 128 of its 242 references that are in the map. First posted on arXiv 9 Jan 2023.

Builds on

128 of 242 references.

References · 128

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  10. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  11. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  12. 2015Spatial Transformer Networkscited by 1,423
38 more
  1. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,420
  2. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  3. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  4. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  5. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  6. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  7. 2021Transformers in Vision: A Surveycited by 1,009
  8. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  9. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  10. 2018Self-Attention Generative Adversarial Networkscited by 809
  11. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  12. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
  13. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  14. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  15. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 588
  16. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  17. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  18. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
  19. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  20. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  21. 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
  22. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
  23. 2021A Survey of Transformerscited by 399
  24. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  25. 2019Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentationcited by 378
  26. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  27. 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
  28. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  29. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  30. 20183D MRI brain tumor segmentation using autoencoder regularizationcited by 324
  31. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 319
  32. 2018Attention Gated Networks: Learning to Leverage Salient Regions in Medical Imagescited by 271
  33. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  34. 2022Brain Imaging Generation with Latent Diffusion Modelscited by 267
  35. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  36. 2021TransFuse: Fusing Transformers and CNNs for Medical Image Segmentationcited by 258
  37. 2022Transformers in Medical Imaging: A Surveycited by 245
  38. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 20; 2025: 21; 2026: 9.

62 papers.

Sort:

By area:

  1. 2023Implicit Neural Representation in Medical Imaging: A Comparative Surveycited by 88
  2. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  3. 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlookcited by 33
  4. 2023Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentationcited by 30
  5. 2024Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysiscited by 23
  6. 2023A Recent Survey of Vision Transformers for Medical Image Segmentationcited by 22
  7. 2023Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospectscited by 22
  8. 2024SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imagingcited by 17
  9. 2024Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A surveycited by 13
  10. 2023DermoSegDiff: A Boundary-aware Segmentation Diffusion Model for Skin Lesion Delineationcited by 12
  11. 2024xLSTM-UNet can be an Effective 2D & 3D Medical Image Segmentation Backbone with Vision-LSTM (ViL) better than its Mamba Counterpartcited by 12
  12. 2023Enhancing Medical Image Segmentation with TransCeption: A Multi-Scale Feature Fusion Approachcited by 11
38 more
  1. 2024Multimodal Data Integration for Precision Oncology: Challenges and Future Directionscited by 10
  2. 2025Circuit Complexity Bounds for Visual Autoregressive Modelcited by 10
  3. 2023Attention Mechanisms in Medical Image Segmentation: A Surveycited by 9
  4. 2024Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastomacited by 9
  5. 2025Recent Advances in Medical Imaging Segmentation: A Surveycited by 8
  6. 2024Tissue Concepts: supervised foundation models in computational pathologycited by 7
  7. 2025Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truthcited by 7
  8. 2023Self-supervised Few-shot Learning for Semantic Segmentation: An Annotation-free Approachcited by 6
  9. 2024Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Modelscited by 6
  10. 2023Self-supervised Semantic Segmentation: Consistency over Transformationcited by 5
  11. 2025Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attentioncited by 5
  12. 2024Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insightscited by 4
  13. 2024LHU-Net: a Lean Hybrid U-Net for Cost-efficient, High-performance Volumetric Segmentationcited by 4
  14. 2024Deep Learning for Pancreas Segmentation: a Systematic Reviewcited by 4
  15. 2024MedVisionLlama: Leveraging Pre-Trained Large Language Model Layers to Enhance Medical Image Segmentationcited by 4
  16. 2025A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastomacited by 4
  17. 2026Meningioma Analysis and Diagnosis using Limited Labeled Samplescited by 4
  18. 2023Unlocking Fine-Grained Details with Wavelet-based High-Frequency Enhancement in Transformerscited by 3
  19. 2024Key Patches Are All You Need: A Multiple Instance Learning Framework For Robust Medical Diagnosiscited by 3
  20. 2025GONet: A Generalizable Deep Learning Model for Glaucoma Detectioncited by 3
  21. 2025A Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Disease Detection from Retinal Fundus Imagescited by 3
  22. 2025MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attentioncited by 3
  23. 2024Integrating Clinical Knowledge Graphs and Gradient-Based Neural Systems for Enhanced Melanoma Diagnosis via the 7-Point Checklistcited by 2
  24. 2024Advances in Transformers for Robotic Applications: A Reviewcited by 2
  25. 2025AutoRad-Lung: A Radiomic-Guided Prompting Autoregressive Vision-Language Model for Lung Nodule Malignancy Predictioncited by 2
  26. 2025WoundAmbit: Bridging State-of-the-Art Semantic Segmentation and Real-World Wound Carecited by 2
  27. 2025Structured Spectral Graph Learning for Anomaly Classification in 3D Chest CT Scanscited by 2
  28. 2024Multi-modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generationcited by 1
  29. 2025Med-R2: Perception and Reflection-driven Complex Reasoning for Medical Report Generationcited by 1
  30. 2026MTFlow: Time-Conditioned Flow Matching for Microtubule Segmentation in Noisy Microscopy Imagescited by 1
  31. 2024Deep Spatiotemporal Clutter Filtering of Transthoracic Echocardiographic Images: Leveraging Contextual Attention and Residual Learningcited by 0
  32. 2024Channel Boosted CNN-Transformer-based Multi-Level and Multi-Scale Nuclei Segmentationcited by 0
  33. 2024Ultrasound-Based AI for COVID-19 Detection: A Comprehensive Review of Public and Private Lung Ultrasound Datasets and Studiescited by 0
  34. 2024Artificial Intelligence in Pediatric Echocardiography: Exploring Challenges, Opportunities, and Clinical Applications with Explainable AI and Federated Learningcited by 0
  35. 2025Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learningcited by 0
  36. 2025MERA: Multimodal and Multiscale Self-Explanatory Model with Considerably Reduced Annotation for Lung Nodule Diagnosiscited by 0
  37. 2025Tetrahedron-Net for Medical Image Registrationcited by 0
  38. 2025DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformercited by 0

50 most cited of 62 All 62 (JSON)