mlsanityData to 7 Oct 2026

COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction

arXiv 2101.04909 ↗First posted 13 Jan 2021cs.CV cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

The rapid spread of COVID-19 cases in recent months has strained hospital resources, making rapid and accurate triage of patients presenting to emergency departments a necessity. Machine learning techniques using clinical data such as chest X-rays have been used to predict which patients are most at risk of deterioration. We consider the task of predicting two types of patient deterioration based on chest X-rays: adverse event deterioration (i.e., transfer to the intensive care unit, intubation, or mortality) and increased oxygen requirements beyond 6 L per day. Due to the relative scarcity of COVID-19 patient data, existing solutions leverage supervised pretraining on related non-COVID images, but this is limited by the differences between the pretraining data and the target COVID-19 patient data. In this paper, we use self-supervised learning based on the momentum contrast (MoCo) method in the pretraining phase to learn more general image representations to use for downstream tasks. We present three results. The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0.742 for predicting an adverse event within 96 hours (compared to 0.703 with supervised pretraining) and an AUC of 0.765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0.749 with supervised pretraining). We then propose a new transformer-based architecture that can process sequences of multiple images for prediction and show that this model can achieve an improved AUC of 0.786 for predicting an adverse event at 96 hours and an AUC of 0.848 for predicting mortalities at 96 hours. A small pilot clinical study suggested that the prediction accuracy of our model is comparable to that of experienced radiologists analyzing the same information.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 13 Jan 2021.

Builds on

18 of 38 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  11. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  12. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
6 more
  1. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  2. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  3. 2016Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learningcited by 209
  4. 2020Deep Learning COVID-19 Features on CXR using Limited Training Data Setscited by 68
  5. 2020An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localizationcited by 43
  6. 2020Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Imagescited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 11; 2023: 6; 2024: 7; 2025: 2; 2026: 2.

36 papers.

Sort:

  1. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  2. 2021Joint Learning of Localized Representations from Medical Images and Reportscited by 81
  3. 2021Self-supervised learning methods and applications in medical imaging analysis: A surveycited by 76
  4. 2021MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretationcited by 52
  5. 2022Transformers in Medical Image Analysis: A Reviewcited by 41
  6. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  7. 2021Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classificationcited by 12
  8. 2022Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographscited by 12
  9. 2021An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysiscited by 10
  10. 2023A Review of Predictive and Contrastive Self-supervised Learning for Medical Imagescited by 10
  11. 2021SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadatacited by 9
  12. 2024Time-to-Event Pretraining for 3D Medical Imagingcited by 7
24 more
  1. 2022CASS: Cross Architectural Self-Supervision for Medical Image Analysiscited by 6
  2. 2022SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Modelcited by 6
  3. 2024MedMAE: A Self-Supervised Backbone for Medical Imaging Taskscited by 5
  4. 2021Performance or Trust? Why Not Both. Deep AUC Maximization with Self-Supervised Learning for COVID-19 Chest X-ray Classificationscited by 4
  5. 2022Improving Classification Model Performance on Chest X-Rays through Lung Segmentationcited by 4
  6. 2024DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studiescited by 4
  7. 2023TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Predictioncited by 3
  8. 2022Self-supervised Model Based on Masked Autoencoders Advance CT Scans Classificationcited by 2
  9. 2023Efficient Representation Learning for Healthcare with Cross-Architectural Self-Supervisioncited by 2
  10. 2023Enhancing Medical Image Segmentation: Optimizing Cross-Entropy Weights and Post-Processing with Autoencoderscited by 2
  11. 2022Unsupervised Feature Clustering Improves Contrastive Representation Learning for Medical Image Segmentationcited by 1
  12. 2022A Trustworthy Framework for Medical Image Analysis with Deep Learningcited by 1
  13. 2024Learning Generalized Medical Image Representations through Image-Graph Contrastive Pretrainingcited by 1
  14. 2025CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detectioncited by 1
  15. 2025STA-Risk: A Deep Dive of Spatio-Temporal Asymmetries for Breast Cancer Risk Predictioncited by 1
  16. 2021Hierarchical Analysis of Visual COVID-19 Features from Chest Radiographscited by 0
  17. 2022Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Taskscited by 0
  18. 2022A Novel Center-based Deep Contrastive Metric Learning Method for the Detection of Polymicrogyria in Pediatric Brain MRIcited by 0
  19. 2023Cross-Architectural Positive Pairs improve the effectiveness of Self-Supervised Learningcited by 0
  20. 2024Self-Supervised Learning Featuring Small-Scale Image Dataset for Treatable Retinal Diseases Classificationcited by 0
  21. 2024Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modelingcited by 0
  22. 2024Multi-modal Masked Siamese Network Improves Chest X-Ray Representation Learningcited by 0
  23. 2026Benchmarking transferability of SSL pretraining to same and different modality segmentation taskscited by 0
  24. 2026Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelinescited by 0