# Vision Transformer based COVID-19 Detection using Chest X-rays

arXiv: https://arxiv.org/abs/2110.04458 · HTML: https://mlsanity.com/paper/2110.04458/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 9 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Can AI help in screening Viral and COVID-19 pneumonia?](https://mlsanity.com/paper/2003.13145/) (2020)

## Built on by (most cited first, 14 of 14)

- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning](https://mlsanity.com/paper/2205.12466/) (2022)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [SwiftSRGAN -- Rethinking Super-Resolution for Efficient and Real-time Inference](https://mlsanity.com/paper/2111.14320/) (2021)
- [Benchmarking Conventional Vision Models on Neuromorphic Fall Detection and Action Recognition Dataset](https://mlsanity.com/paper/2201.12285/) (2022)
- [MFAAN: Unveiling Audio Deepfakes with a Multi-Feature Authenticity Network](https://mlsanity.com/paper/2311.03509/) (2023)
- [Connectional-Style-Guided Contextual Representation Learning for Brain Disease Diagnosis](https://mlsanity.com/paper/2306.05297/) (2023)
- [Adaptive Input-image Normalization for Solving the Mode Collapse Problem in GAN-based X-ray Images](https://mlsanity.com/paper/2309.12245/) (2023)
- [Weakly-supervised Generative Adversarial Networks for medical image classification](https://mlsanity.com/paper/2111.14605/) (2021)
- [RadTex: Learning Efficient Radiograph Representations from Text Reports](https://mlsanity.com/paper/2208.03218/) (2022)
- [CCTCOVID: COVID-19 Detection from Chest X-Ray Images Using Compact Convolutional Transformers](https://mlsanity.com/paper/2209.13399/) (2022)
- [CECT: Controllable Ensemble CNN and Transformer for COVID-19 Image Classification](https://mlsanity.com/paper/2302.02314/) (2023)
- [Advancing Ischemic Stroke Diagnosis: A Novel Two-Stage Approach for Blood Clot Origin Identification](https://mlsanity.com/paper/2304.13775/) (2023)
- [Study of Vision Transformers for Covid-19 Detection from Chest X-rays](https://mlsanity.com/paper/2307.09402/) (2023)

## Abstract (arXiv)

COVID-19 is a global pandemic, and detecting them is a momentous task for medical professionals today due to its rapid mutations. Current methods of examining chest X-rays and CT scan requires profound knowledge and are time consuming, which suggests that it shrinks the precious time of medical practitioners when people's lives are at stake. This study tries to assist this process by achieving state-of-the-art performance in classifying chest X-rays by fine-tuning Vision Transformer(ViT). The proposed approach uses pretrained models, fine-tuned for detecting the presence of COVID-19 disease on chest X-rays. This approach achieves an accuracy score of 97.61%, precision score of 95.34%, recall score of 93.84% and, f1-score of 94.58%. This result signifies the performance of transformer-based models on chest X-ray.