mlsanityData to 14 Oct 2026

Vision Transformers in Medical Computer Vision -- A Contemplative Retrospection

arXiv 2203.15269 ↗First posted 29 Mar 2022eess.IV cs.CV cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Recent escalation in the field of computer vision underpins a huddle of algorithms with the magnificent potential to unravel the information contained within images. These computer vision algorithms are being practised in medical image analysis and are transfiguring the perception and interpretation of Imaging data. Among these algorithms, Vision Transformers are evolved as one of the most contemporary and dominant architectures that are being used in the field of computer vision. These are immensely utilized by a plenty of researchers to perform new as well as former experiments. Here, in this article we investigate the intersection of Vision Transformers and Medical images and proffered an overview of various ViTs based frameworks that are being used by different researchers in order to decipher the obstacles in Medical Computer Vision. We surveyed the application of Vision transformers in different areas of medical computer vision such as image-based disease classification, anatomical structure segmentation, registration, region-based lesion Detection, captioning, report generation, reconstruction using multiple medical imaging modalities that greatly assist in medical diagnosis and hence treatment process. Along with this, we also demystify several imaging modalities used in Medical Computer Vision. Moreover, to get more insight and deeper understanding, self-attention mechanism of transformers is also explained briefly. Conclusively, we also put some light on available data sets, adopted methodology, their performance measures, challenges and their solutions in form of discussion. We hope that this review article will open future directions for researchers in medical computer vision.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 80 of its 239 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

80 of 239 references.

References · 80

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2017Mask R-CNNcited by 7,277
  8. 2016Densely Connected Convolutional Networkscited by 6,720
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  11. 2017Squeeze-and-Excitation Networkscited by 4,789
  12. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
38 more
  1. 2014Going Deeper with Convolutionscited by 4,241
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  4. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  5. 2015Fast R-CNNcited by 2,933
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  9. 2015Spatial Transformer Networkscited by 1,429
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  11. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  12. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,186
  13. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  14. 2021Transformers in Vision: A Surveycited by 1,013
  15. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  16. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  17. 2020A Survey on Visual Transformercited by 654
  18. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
  19. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  20. 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 309
  21. 2020COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Imagescited by 264
  22. 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 260
  23. 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 231
  24. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  25. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  26. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
  27. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 213
  28. 2016Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challengecited by 201
  29. 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 190
  30. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  31. 2018Neural Baby Talkcited by 107
  32. 2020COVID-19 Image Data Collection: Prospective Predictions Are the Futurecited by 100
  33. 2020COVID-CT-Dataset: A CT Scan Dataset about COVID-19cited by 94
  34. 2021TransMed: Transformers Advance Multi-modal Medical Image Classificationcited by 86
  35. 2020BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientscited by 82
  36. 2021U-Net Transformer: Self and Cross Attention for Medical Image Segmentationcited by 81
  37. 2021MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysiscited by 72
  38. 2018HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentationcited by 66

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 3; 2025: 8; 2026: 2.

22 papers.

Sort:

By area:

  1. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 33
  2. 2023Multimodal Sentiment Analysis: A Surveycited by 12
  3. 2023Is attention all you need in medical image analysis? A reviewcited by 12
  4. 2022Recent Progress in Transformer-based Medical Image Analysiscited by 9
  5. 2023Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformerscited by 9
  6. 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
  7. 2024TransUKAN:Computing-Efficient Hybrid KAN-Transformer for Enhanced Medical Image Segmentationcited by 6
  8. 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studiescited by 4
  9. 2025Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomographycited by 4
  10. 2023An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reportscited by 2
  11. 2025Representation Learning of Lab Values via Masked AutoEncoderscited by 2
  12. 2023A Comprehensive Analysis of Blockchain Applications for Securing Computer Vision Systemscited by 1
10 more
  1. 2023Multi-Granularity Framework for Unsupervised Representation Learning of Time Seriescited by 1
  2. 2025SCFormer: Structured Channel-wise Transformer with Cumulative Historical State for Multivariate Time Series Forecastingcited by 1
  3. 2025The Eye as a Window to Systemic Health: A Survey of Retinal Imaging from Classical Techniques to Oculomicscited by 1
  4. 2025SACO: Sequence-Aware Constrained Optimization Framework for Coupon Distribution in E-commercecited by 1
  5. 2024Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessmentcited by 0
  6. 2025ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Acceleratorscited by 0
  7. 2025ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognitioncited by 0
  8. 2025Towards Foundation Models for Cryo-ET Subtomogram Analysiscited by 0
  9. 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentationcited by 0
  10. 2026Do emulated quantum circuits change what CNNs look at? Performance and explainability comparison in medical image classificationcited by 0