mlsanityData to 7 Oct 2026

Recent Progress in Transformer-based Medical Image Analysis

arXiv 2208.06643 ↗First posted 13 Aug 2022eess.IV cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structures of the transformer. After that, we depict the recent progress of the transformer in the field of MIA. We organize the applications in a sequence of different tasks, including classification, segmentation, captioning, registration, detection, enhancement, localization, and synthesis. The mainstream classification and segmentation tasks are further divided into eleven medical image modalities. A large number of experiments studied in this review illustrate that the transformer-based method outperforms existing methods through comparisons with multiple evaluation metrics. Finally, we discuss the open challenges and future opportunities in this field. This task-modality review with the latest contents, detailed information, and comprehensive comparison may greatly benefit the broad MIA community.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 154 of its 444 references that are in the map. First posted on arXiv 13 Aug 2022.

Builds on

154 of 444 references.

References · 154

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2017Mask R-CNNcited by 7,259
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2016Densely Connected Convolutional Networkscited by 6,691
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
38 more
  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2022A ConvNet for the 2020scited by 3,545
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  9. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  10. 2017Non-local Neural Networkscited by 2,290
  11. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  12. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  13. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  14. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  15. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  16. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  17. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  18. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  19. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  20. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  21. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
  22. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  23. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  24. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  25. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  26. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  27. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  28. 2021The Medical Segmentation Decathloncited by 620
  29. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  30. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 588
  31. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
  32. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  33. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  34. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
  35. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  36. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 473
  37. 2018HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Imagescited by 387
  38. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 371

50 most cited of 154 All 154 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 6; 2025: 1.

9 papers.

  1. 2023A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Taskscited by 36
  2. 2023MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysiscited by 4
  3. 2024A Comparative Study of CNN, ResNet, and Vision Transformers for Multi-Classification of Chest Diseasescited by 3
  4. 2024Segmenting Medical Images with Limited Datacited by 3
  5. 2024Enhancing Skin Disease Classification Leveraging Transformer-based Deep Learning Architectures and Explainable AIcited by 1
  6. 2024Visual Question Answering in Ophthalmology: A Progressive and Practical Perspectivecited by 1
  7. 2024Advancing Efficient Brain Tumor Multi-Class Classification -- New Insights from the Vision Mamba Model in Transfer Learningcited by 1
  8. 2024Adaptive Query Prompting for Multi-Domain Landmark Detectioncited by 0
  9. 2025MTMed3D: A Multi-Task Transformer-Based Model for 3D Medical Imagingcited by 0