An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports
Not on the map. Why?
Abstract (arXiv)
The pandemic resulted in vast repositories of unstructured data, including radiology reports, due to increased medical examinations. Previous research on automated diagnosis of COVID-19 primarily focuses on X-ray images, despite their lower precision compared to computed tomography (CT) scans. In this work, we leverage unstructured data from a hospital and harness the fine-grained details offered by CT scans to perform zero-shot multi-label classification based on contrastive visual language learning. In collaboration with human experts, we investigate the effectiveness of multiple zero-shot models that aid radiologists in detecting pulmonary embolisms and identifying intricate lung details like ground glass opacities and consolidations. Our empirical analysis provides an overview of the possible solutions to target such fine-grained tasks, so far overlooked in the medical multimodal pretraining literature. Our investigation promises future advancements in the medical image analysis community by addressing some challenges associated with unstructured data and fine-grained multi-label classification.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 4 Sep 2023.
Builds on
30 of 59 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
18 more
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
- 2019Publicly Available Clinical BERT Embeddingscited by 995
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
- 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
- 2020Understanding the Behaviour of Contrastive Losscited by 397
- 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
- 2020COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Imagescited by 264
- 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
- 2020How Well Do Self-Supervised Models Transfer?cited by 213
- 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 132
- 2021When Does Contrastive Visual Representation Learning Work?cited by 92
- 2021Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?cited by 85
- 2019Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilationcited by 30
- 2021COVID-VIT: Classification of COVID-19 from CT chest images based on vision transformer modelscited by 27
- 2022Vision Transformers in Medical Computer Vision -- A Contemplative Retrospectioncited by 22
- 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.