mlsanityData to 7 Oct 2026

MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models

arXiv 2010.05352 ↗First posted 11 Oct 2020cs.CV cs.AI cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Contrastive learning is a form of self-supervision that can leverage unlabeled data to produce pretrained models. While contrastive learning has demonstrated promising results on natural image classification tasks, its application to medical imaging tasks like chest X-ray interpretation has been limited. In this work, we propose MoCo-CXR, which is an adaptation of the contrastive learning method Momentum Contrast (MoCo), to produce models with better representations and initializations for the detection of pathologies in chest X-rays. In detecting pleural effusion, we find that linear models trained on MoCo-CXR-pretrained representations outperform those without MoCo-CXR-pretrained representations, indicating that MoCo-CXR-pretrained representations are of higher-quality. End-to-end fine-tuning experiments reveal that a model initialized via MoCo-CXR-pretraining outperforms its non-MoCo-CXR-pretrained counterpart. We find that MoCo-CXR-pretraining provides the most benefit with limited labeled training data. Finally, we demonstrate similar results on a target Tuberculosis dataset unseen during pretraining, indicating that MoCo-CXR-pretraining endows models with representations and transferability that can be applied across chest X-ray datasets and tasks.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 14; 2023: 13; 2024: 6; 2025: 3; 2026: 3.

61 papers.

Sort:

By area:

  1. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  2. 2021When Does Contrastive Visual Representation Learning Work?cited by 92
  3. 2021Joint Learning of Localized Representations from Medical Images and Reportscited by 81
  4. 2021Self-supervised learning methods and applications in medical imaging analysis: A surveycited by 76
  5. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 59
  6. 2024FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Modelscited by 47
  7. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  8. 20213KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentationscited by 36
  9. 2021Federated Contrastive Learning for Decentralized Unlabeled Medical Imagescited by 33
  10. 2021How Transferable Are Self-supervised Features in Medical Image Classification Tasks?cited by 32
  11. 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
  12. 2021Intermediate Layers Matter in Momentum Contrastive Self Supervised Learningcited by 19
38 more
  1. 2021Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learningcited by 19
  2. 2021Self-supervised deep convolutional neural network for chest X-ray classificationcited by 16
  3. 2024Improving Medical Multi-modal Contrastive Learning with Expert Annotationscited by 16
  4. 2021Improving Self-supervised Learning with Hardness-aware Dynamic Curriculum Learning: An Application to Digital Pathologycited by 13
  5. 2022Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographscited by 12
  6. 2023A Review of Predictive and Contrastive Self-supervised Learning for Medical Imagescited by 10
  7. 2021SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadatacited by 9
  8. 2021CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancercited by 9
  9. 2022Anatomy-Aware Contrastive Representation Learning for Fetal Ultrasoundcited by 8
  10. 2023Clinically Labeled Contrastive Learning for OCT Biomarker Classificationcited by 8
  11. 2023Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosiscited by 7
  12. 2021Deep AUC Maximization for Medical Image Classification: Challenges and Opportunitiescited by 6
  13. 2021On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathycited by 5
  14. 2022Interpretable Prediction of Lung Squamous Cell Carcinoma Recurrence With Self-supervised Learningcited by 5
  15. 2022Contrastive learning-based pretraining improves representation and transferability of diabetic retinopathy classification modelscited by 5
  16. 2022Keep Your Friends Close & Enemies Farther: Debiasing Contrastive Learning with Spatial Priors in 3D Radiology Imagescited by 5
  17. 2022Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learningcited by 5
  18. 2024Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopycited by 5
  19. 2021MedSelect: Selective Labeling for Medical Image Classification Combining Meta-Learning with Deep Reinforcement Learningcited by 4
  20. 2021Performance or Trust? Why Not Both. Deep AUC Maximization with Self-Supervised Learning for COVID-19 Chest X-ray Classificationscited by 4
  21. 2022Can we Adopt Self-supervised Pretraining for Chest X-Rays?cited by 4
  22. 2023MEDIMP: 3D Medical Images with clinical Prompts from limited tabular data for renal transplantationcited by 4
  23. 2023Learning Better Contrastive View from Radiologist's Gazecited by 4
  24. 2021Unsupervised Local Discrimination for Medical Imagescited by 3
  25. 2022Self-Supervision and Multi-Task Learning: Challenges in Fine-Grained COVID-19 Multi-Class Classification from Chest X-rayscited by 3
  26. 2022From Labels to Priors in Capsule Endoscopy: A Prior Guided Approach for Improving Generalization with Few Labelscited by 3
  27. 2023MoCo-Transfer: Investigating out-of-distribution contrastive learning for limited-data domainscited by 3
  28. 2023A Self Supervised StyleGAN for Image Annotation and Classification with Extremely Limited Labelscited by 3
  29. 2024OTCXR: Rethinking Self-supervised Alignment using Optimal Transport for Chest X-ray Analysiscited by 3
  30. 2023Exploring Self-Supervised Representation Learning For Low-Resource Medical Image Analysiscited by 2
  31. 2024CoBooM: Codebook Guided Bootstrapping for Medical Image Representation Learningcited by 2
  32. 2021Towards to Robust and Generalized Medical Image Segmentation Frameworkcited by 1
  33. 2022Unsupervised Feature Clustering Improves Contrastive Representation Learning for Medical Image Segmentationcited by 1
  34. 2022A Domain-specific Perceptual Metric via Contrastive Self-supervised Representation: Applications on Natural and Medical Imagescited by 1
  35. 2022A Trustworthy Framework for Medical Image Analysis with Deep Learningcited by 1
  36. 2023DrasCLR: A Self-supervised Framework of Learning Disease-related and Anatomy-specific Representation for 3D Medical Imagescited by 1
  37. 2023Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysiscited by 1
  38. 2024Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inferencecited by 1

50 most cited of 61 All 61 (JSON)