# COVID-VIT: Classification of COVID-19 from CT chest images based on vision transformer models arXiv: https://arxiv.org/abs/2107.01682 ยท HTML: https://mlsanity.com/paper/2107.01682/ ## Key facts - Cited by 27 papers in the map (data through 14 Oct 2026). - Builds on 6 of its 20 references that are in the map. - First posted on arXiv 4 Jul 2021. ## Builds on (most cited first, 6 of 6) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [COVID-CT-Dataset: A CT Scan Dataset about COVID-19](https://mlsanity.com/paper/2003.13865/) (2020) - [Deep Transparent Prediction through Latent Representation Analysis](https://mlsanity.com/paper/2009.07044/) (2020) - [MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis](https://mlsanity.com/paper/2106.07524/) (2021) ## Built on by (most cited first, 27 of 27) - [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022) - [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022) - [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023) - [Large-scale Long-tailed Disease Diagnosis on Radiology Images](https://mlsanity.com/paper/2312.16151/) (2023) - [A survey on attention mechanisms for medical applications: are we moving towards better algorithms?](https://mlsanity.com/paper/2204.12406/) (2022) - [HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images](https://mlsanity.com/paper/2205.08390/) (2022) - [Multi-modal Deep Learning](https://mlsanity.com/paper/2403.03385/) (2024) - [JointViT: Modeling Oxygen Saturation Levels with Joint Supervision on Long-Tailed OCTA](https://mlsanity.com/paper/2404.11525/) (2024) - [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022) - [RadCLIP: Enhancing Radiologic Image Analysis through Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2403.09948/) (2024) - [CMC v2: Towards More Accurate COVID-19 Detection with Discriminative Video Priors](https://mlsanity.com/paper/2211.14557/) (2022) - [Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care](https://mlsanity.com/paper/2309.00252/) (2023) - [MultiNet with Transformers: A Model for Cancer Diagnosis Using Images](https://mlsanity.com/paper/2301.09007/) (2023) - [ADAPT: Alzheimer Diagnosis through Adaptive Profiling Transformers](https://mlsanity.com/paper/2401.06349/) (2024) - [3D unsupervised anomaly detection and localization through virtual multi-view projection and reconstruction: Clinical validation on low-dose chest computed tomography](https://mlsanity.com/paper/2206.13385/) (2022) - [Computer Vision for Transit Travel Time Prediction: An End-to-End Framework Using Roadside Urban Imagery](https://mlsanity.com/paper/2211.12322/) (2022) - [An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports](https://mlsanity.com/paper/2309.01740/) (2023) - [Improving diagnosis and prognosis of lung cancer using vision transformers: A scoping review](https://mlsanity.com/paper/2309.02783/) (2023) - [CAE-Transformer: Transformer-based Model to Predict Invasiveness of Lung Adenocarcinoma Subsolid Nodules from Non-thin Section 3D CT Scans](https://mlsanity.com/paper/2110.08721/) (2021) - [Spatio-Temporal Hybrid Fusion of CAE and SWIn Transformers for Lung Cancer Malignancy Prediction](https://mlsanity.com/paper/2210.15297/) (2022) - [Connectional-Style-Guided Contextual Representation Learning for Brain Disease Diagnosis](https://mlsanity.com/paper/2306.05297/) (2023) - [Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis](https://mlsanity.com/paper/2508.04429/) (2025) - [VariViT: A Vision Transformer for Variable Image Sizes](https://mlsanity.com/paper/2602.14615/) (2026) - [Res-Dense Net for 3D Covid Chest CT-scan classification](https://mlsanity.com/paper/2208.04613/) (2022) - [Enhancing COVID-19 Diagnosis through Vision Transformer-Based Analysis of Chest X-ray Images](https://mlsanity.com/paper/2306.06914/) (2023) - [PneumoLLM: Harnessing the Power of Large Language Model for Pneumoconiosis Diagnosis](https://mlsanity.com/paper/2312.03490/) (2023) - [JDCNet: Confidence-Gated Privileged-Modality Distillation for Cost-Preserving X-ray Inference](https://mlsanity.com/paper/2603.29167/) (2026) ## Abstract (arXiv) This paper is responding to the MIA-COV19 challenge to classify COVID from non-COVID based on CT lung images. The COVID-19 virus has devastated the world in the last eighteen months by infecting more than 182 million people and causing over 3.9 million deaths. The overarching aim is to predict the diagnosis of the COVID-19 virus from chest radiographs, through the development of explainable vision transformer deep learning techniques, leading to population screening in a more rapid, accurate and transparent way. In this competition, there are 5381 three-dimensional (3D) datasets in total, including 1552 for training, 374 for evaluation and 3455 for testing. While most of the data volumes are in axial view, there are a number of subjects' data are in coronal or sagittal views with 1 or 2 slices are in axial view. Hence, while 3D data based classification is investigated, in this competition, 2D images remains the main focus. Two deep learning methods are studied, which are vision transformer (ViT) based on attention models and DenseNet that is built upon conventional convolutional neural network (CNN). Initial evaluation results based on validation datasets whereby the ground truth is known indicate that ViT performs better than DenseNet with F1 scores being 0.76 and 0.72 respectively. Codes are available at GitHub at .