# Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning

arXiv: https://arxiv.org/abs/2405.01469 · HTML: https://mlsanity.com/paper/2405.01469/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Exploring scalable medical image encoders beyond text supervision](https://mlsanity.com/paper/2401.10815/) (2024)
- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports](https://mlsanity.com/paper/2111.03452/) (2021)
- [Improving the Fairness of Chest X-ray Classifiers](https://mlsanity.com/paper/2203.12609/) (2022)
- [Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors](https://mlsanity.com/paper/2210.06340/) (2022)
- [Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding](https://mlsanity.com/paper/2210.03416/) (2022)
- [Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling](https://mlsanity.com/paper/2306.03934/) (2023)
- [VinDr-RibCXR: A Benchmark Dataset for Automatic Segmentation and Labeling of Individual Ribs on Chest X-rays](https://mlsanity.com/paper/2107.01327/) (2021)
- [UniChest: Conquer-and-Divide Pre-training for Multi-Source Chest X-Ray Classification](https://mlsanity.com/paper/2312.11038/) (2023)
- [Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms](https://mlsanity.com/paper/2201.07856/) (2022)
- [Foundation Ark: Accruing and Reusing Knowledge for Superior and Robust Performance](https://mlsanity.com/paper/2310.09507/) (2023)
- [POLCOVID: a multicenter multiclass chest X-ray database (Poland, 2020-2021)](https://mlsanity.com/paper/2211.16359/) (2022)

## Built on by (most cited first, 18 of 18)

- [Cluster and Predict Latent Patches for Improved Masked Image Modeling](https://mlsanity.com/paper/2502.08769/) (2025)
- [ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts](https://mlsanity.com/paper/2406.10973/) (2024)
- [Robust image representations with counterfactual contrastive learning](https://mlsanity.com/paper/2409.10365/) (2024)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Mask of truth: model sensitivity to unexpected regions of medical images](https://mlsanity.com/paper/2412.04030/) (2024)
- [Chest X-ray Foundation Model with Global and Local Representations Integration](https://mlsanity.com/paper/2502.05142/) (2025)
- [Decipher-MR: A Vision-Language Foundation Model for 3D MRI Representations](https://mlsanity.com/paper/2509.21249/) (2025)
- [Data Scaling Laws for Radiology Foundation Models](https://mlsanity.com/paper/2509.12818/) (2025)
- [DiNO-Diffusion. Scaling Medical Diffusion via Self-Supervised Pre-Training](https://mlsanity.com/paper/2407.11594/) (2024)
- [Multi Anatomy X-Ray Foundation Model](https://mlsanity.com/paper/2509.12146/) (2025)
- [Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation](https://mlsanity.com/paper/2603.16876/) (2026)
- [Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift](https://mlsanity.com/paper/2607.10358/) (2026)
- [Unsupervised Parameter Efficient Source-free Post-pretraining](https://mlsanity.com/paper/2502.21313/) (2025)
- [Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis](https://mlsanity.com/paper/2504.16047/) (2025)
- [Capability neq Interpretability: Human Interpretability of Vision Foundation Models](https://mlsanity.com/paper/2605.20337/) (2026)
- [Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Have](https://mlsanity.com/paper/2606.05107/) (2026)
- [Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning](https://mlsanity.com/paper/2607.00850/) (2026)
- [B-MIM: Biased Masked Image Modeling for Generalizable Segmentation of Fine-Grained Anatomical Structures](https://mlsanity.com/paper/2608.24364/) (2026)

## Abstract (arXiv)

AI Foundation models are gaining traction in various applications, including medical fields like radiology. However, medical foundation models are often tested on limited tasks, leaving their generalisability and biases unexplored. We present RayDINO, a large visual encoder trained by self-supervision on 873k chest X-rays. We compare RayDINO to previous state-of-the-art models across nine radiology tasks, from classification and dense segmentation to text generation, and provide an in depth analysis of population, age and sex biases of our model. Our findings suggest that self-supervision allows patient-centric AI proving useful in clinical workflows and interpreting X-rays holistically. With RayDINO and small task-specific adapters, we reach state-of-the-art results and improve generalization to unseen populations while mitigating bias, illustrating the true promise of foundation models: versatility and robustness.