# MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models

arXiv: https://arxiv.org/abs/2010.05352 · HTML: https://mlsanity.com/paper/2010.05352/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 30 of 61)

- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Joint Learning of Localized Representations from Medical Images and Reports](https://mlsanity.com/paper/2112.02889/) (2021)
- [Self-supervised learning methods and applications in medical imaging analysis: A survey](https://mlsanity.com/paper/2109.08685/) (2021)
- [Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts](https://mlsanity.com/paper/2109.04379/) (2021)
- [FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models](https://mlsanity.com/paper/2407.00983/) (2024)
- [Robust and Efficient Medical Imaging with Self-Supervision](https://mlsanity.com/paper/2205.09723/) (2022)
- [3KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentations](https://mlsanity.com/paper/2106.04452/) (2021)
- [Federated Contrastive Learning for Decentralized Unlabeled Medical Images](https://mlsanity.com/paper/2109.07504/) (2021)
- [How Transferable Are Self-supervised Features in Medical Image Classification Tasks?](https://mlsanity.com/paper/2108.10048/) (2021)
- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning](https://mlsanity.com/paper/2110.14805/) (2021)
- [Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning](https://mlsanity.com/paper/2111.10827/) (2021)
- [Self-supervised deep convolutional neural network for chest X-ray classification](https://mlsanity.com/paper/2103.03055/) (2021)
- [Improving Medical Multi-modal Contrastive Learning with Expert Annotations](https://mlsanity.com/paper/2403.10153/) (2024)
- [Improving Self-supervised Learning with Hardness-aware Dynamic Curriculum Learning: An Application to Digital Pathology](https://mlsanity.com/paper/2108.07183/) (2021)
- [Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs](https://mlsanity.com/paper/2205.04282/) (2022)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadata](https://mlsanity.com/paper/2110.14787/) (2021)
- [CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancer](https://mlsanity.com/paper/2112.01330/) (2021)
- [Clinically Labeled Contrastive Learning for OCT Biomarker Classification](https://mlsanity.com/paper/2305.15154/) (2023)
- [Anatomy-Aware Contrastive Representation Learning for Fetal Ultrasound](https://mlsanity.com/paper/2208.10642/) (2022)
- [Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis](https://mlsanity.com/paper/2312.06069/) (2023)
- [Deep AUC Maximization for Medical Image Classification: Challenges and Opportunities](https://mlsanity.com/paper/2111.02400/) (2021)
- [On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy](https://mlsanity.com/paper/2106.13497/) (2021)
- [Interpretable Prediction of Lung Squamous Cell Carcinoma Recurrence With Self-supervised Learning](https://mlsanity.com/paper/2203.12204/) (2022)
- [Contrastive learning-based pretraining improves representation and transferability of diabetic retinopathy classification models](https://mlsanity.com/paper/2208.11563/) (2022)
- [Keep Your Friends Close & Enemies Farther: Debiasing Contrastive Learning with Spatial Priors in 3D Radiology Images](https://mlsanity.com/paper/2211.08643/) (2022)
- [Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning](https://mlsanity.com/paper/2212.04097/) (2022)
- [Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy](https://mlsanity.com/paper/2402.18286/) (2024)

## Abstract (arXiv)

Contrastive learning is a form of self-supervision that can leverage unlabeled data to produce pretrained models. While contrastive learning has demonstrated promising results on natural image classification tasks, its application to medical imaging tasks like chest X-ray interpretation has been limited. In this work, we propose MoCo-CXR, which is an adaptation of the contrastive learning method Momentum Contrast (MoCo), to produce models with better representations and initializations for the detection of pathologies in chest X-rays. In detecting pleural effusion, we find that linear models trained on MoCo-CXR-pretrained representations outperform those without MoCo-CXR-pretrained representations, indicating that MoCo-CXR-pretrained representations are of higher-quality. End-to-end fine-tuning experiments reveal that a model initialized via MoCo-CXR-pretraining outperforms its non-MoCo-CXR-pretrained counterpart. We find that MoCo-CXR-pretraining provides the most benefit with limited labeled training data. Finally, we demonstrate similar results on a target Tuberculosis dataset unseen during pretraining, indicating that MoCo-CXR-pretraining endows models with representations and transferability that can be applied across chest X-ray datasets and tasks.