Vision Transformers in Medical Computer Vision -- A Contemplative Retrospection
Not on the map. Why?
Abstract (arXiv)
Recent escalation in the field of computer vision underpins a huddle of algorithms with the magnificent potential to unravel the information contained within images. These computer vision algorithms are being practised in medical image analysis and are transfiguring the perception and interpretation of Imaging data. Among these algorithms, Vision Transformers are evolved as one of the most contemporary and dominant architectures that are being used in the field of computer vision. These are immensely utilized by a plenty of researchers to perform new as well as former experiments. Here, in this article we investigate the intersection of Vision Transformers and Medical images and proffered an overview of various ViTs based frameworks that are being used by different researchers in order to decipher the obstacles in Medical Computer Vision. We surveyed the application of Vision transformers in different areas of medical computer vision such as image-based disease classification, anatomical structure segmentation, registration, region-based lesion Detection, captioning, report generation, reconstruction using multiple medical imaging modalities that greatly assist in medical diagnosis and hence treatment process. Along with this, we also demystify several imaging modalities used in Medical Computer Vision. Moreover, to get more insight and deeper understanding, self-attention mechanism of transformers is also explained briefly. Conclusively, we also put some light on available data sets, adopted methodology, their performance measures, challenges and their solutions in form of discussion. We hope that this review article will open future directions for researchers in medical computer vision.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 80 of its 239 references that are in the map. First posted on arXiv 29 Mar 2022.
Builds on
80 of 239 references.
References · 80
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2017Mask R-CNNcited by 7,277
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
38 more
- 2014Going Deeper with Convolutionscited by 4,241
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
- 2015SSD: Single Shot MultiBox Detectorcited by 3,130
- 2015Fast R-CNNcited by 2,933
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
- 2015Spatial Transformer Networkscited by 1,429
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
- 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
- 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,186
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
- 2021Transformers in Vision: A Surveycited by 1,013
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
- 2020A Survey on Visual Transformercited by 654
- 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
- 2019Meshed-Memory Transformer for Image Captioningcited by 343
- 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 309
- 2020COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Imagescited by 264
- 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 260
- 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 231
- 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
- 2017On the Automatic Generation of Medical Imaging Reportscited by 216
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
- 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 213
- 2016Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challengecited by 201
- 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 190
- 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
- 2018Neural Baby Talkcited by 107
- 2020COVID-19 Image Data Collection: Prospective Predictions Are the Futurecited by 100
- 2020COVID-CT-Dataset: A CT Scan Dataset about COVID-19cited by 94
- 2021TransMed: Transformers Advance Multi-modal Medical Image Classificationcited by 86
- 2020BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientscited by 82
- 2021U-Net Transformer: Self and Cross Attention for Medical Image Segmentationcited by 81
- 2021MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysiscited by 72
- 2018HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentationcited by 66
50 most cited of 80 All 80 (JSON)
Built on by
Citing papers per year: 2022: 3; 2023: 6; 2024: 3; 2025: 8; 2026: 2.
22 papers.
Sort:
By area:
- 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 33
- 2023Multimodal Sentiment Analysis: A Surveycited by 12
- 2023Is attention all you need in medical image analysis? A reviewcited by 12
- 2022Recent Progress in Transformer-based Medical Image Analysiscited by 9
- 2023Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformerscited by 9
- 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
- 2024TransUKAN:Computing-Efficient Hybrid KAN-Transformer for Enhanced Medical Image Segmentationcited by 6
- 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studiescited by 4
- 2025Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomographycited by 4
- 2023An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reportscited by 2
- 2025Representation Learning of Lab Values via Masked AutoEncoderscited by 2
- 2023A Comprehensive Analysis of Blockchain Applications for Securing Computer Vision Systemscited by 1
10 more
- 2023Multi-Granularity Framework for Unsupervised Representation Learning of Time Seriescited by 1
- 2025SCFormer: Structured Channel-wise Transformer with Cumulative Historical State for Multivariate Time Series Forecastingcited by 1
- 2025The Eye as a Window to Systemic Health: A Survey of Retinal Imaging from Classical Techniques to Oculomicscited by 1
- 2025SACO: Sequence-Aware Constrained Optimization Framework for Coupon Distribution in E-commercecited by 1
- 2024Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessmentcited by 0
- 2025ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Acceleratorscited by 0
- 2025ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognitioncited by 0
- 2025Towards Foundation Models for Cryo-ET Subtomogram Analysiscited by 0
- 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentationcited by 0
- 2026Do emulated quantum circuits change what CNNs look at? Performance and explainability comparison in medical image classificationcited by 0
- 2026Do emulated quantum circuits change what CNNs look at? Performance and explainability comparison in medical image classification23 Jul 2026
- 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentation23 Feb 2026
- 2025Towards Foundation Models for Cryo-ET Subtomogram Analysis29 Sep 2025
- 2025SACO: Sequence-Aware Constrained Optimization Framework for Coupon Distribution in E-commerce8 Aug 2025
- 2025ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition6 Jul 2025
- 2025The Eye as a Window to Systemic Health: A Survey of Retinal Imaging from Classical Techniques to Oculomics6 May 2025
- 2025SCFormer: Structured Channel-wise Transformer with Cumulative Historical State for Multivariate Time Series Forecasting5 May 2025
- 2025ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Accelerators15 Apr 2025
- 2025Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomography4 Apr 2025
- 2025Representation Learning of Lab Values via Masked AutoEncoders5 Jan 2025
- 2024TransUKAN:Computing-Efficient Hybrid KAN-Transformer for Enhanced Medical Image Segmentation23 Sep 2024
- 2024Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessment9 Jun 2024
10 more
- 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studies22 Feb 2024
- 2023Multi-Granularity Framework for Unsupervised Representation Learning of Time Series12 Dec 2023
- 2023An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports4 Sep 2023
- 2023Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers18 Aug 2023
- 2023Is attention all you need in medical image analysis? A review24 Jul 2023
- 2023A Comprehensive Analysis of Blockchain Applications for Securing Computer Vision Systems13 Jul 2023
- 2023Multimodal Sentiment Analysis: A Survey12 May 2023
- 2022Vision Transformers in Medical Imaging: A Review18 Nov 2022
- 2022Recent Progress in Transformer-based Medical Image Analysis13 Aug 2022
- 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives2 Jun 2022
- 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 33
- 2023Multimodal Sentiment Analysis: A Surveycited by 12
- 2023Is attention all you need in medical image analysis? A reviewcited by 12
- 2022Recent Progress in Transformer-based Medical Image Analysiscited by 9
- 2023Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformerscited by 9
- 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
- 2024TransUKAN:Computing-Efficient Hybrid KAN-Transformer for Enhanced Medical Image Segmentationcited by 6
- 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studiescited by 4
- 2023An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reportscited by 2
- 2025Representation Learning of Lab Values via Masked AutoEncoderscited by 2
- 2023A Comprehensive Analysis of Blockchain Applications for Securing Computer Vision Systemscited by 1
- 2023Multi-Granularity Framework for Unsupervised Representation Learning of Time Seriescited by 1
12 most cited of 21 All 21 (JSON)
- 2026Do emulated quantum circuits change what CNNs look at? Performance and explainability comparison in medical image classification23 Jul 2026
- 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentation23 Feb 2026
- 2025Towards Foundation Models for Cryo-ET Subtomogram Analysis29 Sep 2025
- 2025SACO: Sequence-Aware Constrained Optimization Framework for Coupon Distribution in E-commerce8 Aug 2025
- 2025ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition6 Jul 2025
- 2025The Eye as a Window to Systemic Health: A Survey of Retinal Imaging from Classical Techniques to Oculomics6 May 2025
- 2025SCFormer: Structured Channel-wise Transformer with Cumulative Historical State for Multivariate Time Series Forecasting5 May 2025
- 2025ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Accelerators15 Apr 2025
- 2025Representation Learning of Lab Values via Masked AutoEncoders5 Jan 2025
- 2024TransUKAN:Computing-Efficient Hybrid KAN-Transformer for Enhanced Medical Image Segmentation23 Sep 2024
- 2024Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessment9 Jun 2024
- 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studies22 Feb 2024
12 newest of 21 All 21 (JSON)