# Enhancing Network Initialization for Medical AI Models Using Large-Scale, Unlabeled Natural Images

arXiv: https://arxiv.org/abs/2308.07688 · HTML: https://mlsanity.com/paper/2308.07688/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 75 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [NegBio: a high-performance tool for negation and uncertainty detection in radiology reports](https://mlsanity.com/paper/1712.05898/) (2017)
- [Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation](https://mlsanity.com/paper/1603.08486/) (2016)
- [CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation](https://mlsanity.com/paper/2101.06871/) (2021)
- [Weighted Ensemble Self-Supervised Learning](https://mlsanity.com/paper/2211.09981/) (2022)
- [Private, fair and accurate: Training large-scale, privacy-preserving AI models in medical imaging](https://mlsanity.com/paper/2302.01622/) (2023)
- [Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels](https://mlsanity.com/paper/2211.13606/) (2022)
- [Preserving privacy in domain transfer of medical AI models comes at no performance costs: The integral role of differential privacy](https://mlsanity.com/paper/2306.06503/) (2023)

## Built on by (most cited first, 11 of 11)

- [EVA-X: A Foundation Model for General Chest X-ray Analysis with Self-supervised Learning](https://mlsanity.com/paper/2405.05237/) (2024)
- [Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2](https://mlsanity.com/paper/2411.15802/) (2024)
- [Boosting multi-demographic federated learning for chest radiograph analysis using general-purpose self-supervised representations](https://mlsanity.com/paper/2504.08584/) (2025)
- [Meningioma Analysis and Diagnosis using Limited Labeled Samples](https://mlsanity.com/paper/2602.13335/) (2026)
- [Resolution scaling governs DINOv3 transfer performance in chest radiograph classification](https://mlsanity.com/paper/2510.07191/) (2025)
- [The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis](https://mlsanity.com/paper/2601.19618/) (2026)
- [Topology Only Pre-Training: Towards Generalised Multi-Domain Graph Models](https://mlsanity.com/paper/2311.03976/) (2023)
- [DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability](https://mlsanity.com/paper/2407.18100/) (2024)
- [Multi Anatomy X-Ray Foundation Model](https://mlsanity.com/paper/2509.12146/) (2025)
- [Cross-modal linkage risk in clinical vision-language models](https://mlsanity.com/paper/2606.02276/) (2026)
- [CANDOR: Chance-Calibrated Neighborhood Discordance in Frozen Encoders for Medical Imaging](https://mlsanity.com/paper/2607.18451/) (2026)

## Abstract (arXiv)

Pre-training datasets, like ImageNet, have become the gold standard in medical image analysis. However, the emergence of self-supervised learning (SSL), which leverages unlabeled data to learn robust features, presents an opportunity to bypass the intensive labeling process. In this study, we explored if SSL for pre-training on non-medical images can be applied to chest radiographs and how it compares to supervised pre-training on non-medical images and on medical images. We utilized a vision transformer and initialized its weights based on (i) SSL pre-training on natural images (DINOv2), (ii) SL pre-training on natural images (ImageNet dataset), and (iii) SL pre-training on chest radiographs from the MIMIC-CXR database. We tested our approach on over 800,000 chest radiographs from six large global datasets, diagnosing more than 20 different imaging findings. Our SSL pre-training on curated images not only outperformed ImageNet-based pre-training (P<0.001 for all datasets) but, in certain cases, also exceeded SL on the MIMIC-CXR dataset. Our findings suggest that selecting the right pre-training strategy, especially with SSL, can be pivotal for improving artificial intelligence (AI)'s diagnostic accuracy in medical imaging. By demonstrating the promise of SSL in chest radiograph analysis, we underline a transformative shift towards more efficient and accurate AI models in medical imaging.