mlsanityData to 7 Oct 2026

Can we Adopt Self-supervised Pretraining for Chest X-Rays?

arXiv 2211.12931 ↗First posted 23 Nov 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Chest radiograph (or Chest X-Ray, CXR) is a popular medical imaging modality that is used by radiologists across the world to diagnose heart or lung conditions. Over the last decade, Convolutional Neural Networks (CNN), have seen success in identifying pathologies in CXR images. Typically, these CNNs are pretrained on the standard ImageNet classification task, but this assumes availability of large-scale annotated datasets. In this work, we analyze the utility of pretraining on unlabeled ImageNet or Chest X-Ray (CXR) datasets using various algorithms and in multiple settings. Some findings of our work include: (i) supervised training with labeled ImageNet learns strong representations that are hard to beat; (ii) self-supervised pretraining on ImageNet (~1M images) shows performance similar to self-supervised pretraining on a CXR dataset (~100K images); and (iii) the CNN trained on supervised ImageNet can be trained further with self-supervised CXR images leading to improvements, especially when the downstream dataset is on the order of a few thousand images.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

16 of 33 references.

References · 16

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  8. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  9. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  10. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  11. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  12. 2021Self-Supervised Pretraining Improves Self-Supervised Pretrainingcited by 81
4 more
  1. 2020MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Modelscited by 61
  2. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  3. 2021MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretationcited by 52
  4. 2021Self-supervised deep convolutional neural network for chest X-ray classificationcited by 16

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023How Can We Tame the Long-Tail of Chest X-ray Datasets?cited by 2
  2. 2024Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-trainingcited by 2
  3. 2023Generalized Cross-domain Multi-label Few-shot Learning for Chest X-rayscited by 1
  4. 2024Multi-modal Masked Siamese Network Improves Chest X-Ray Representation Learningcited by 0