# Feature-level Site Leakage Reduction for Cross-Hospital Chest X-ray Transfer via Self-Supervised Learning

arXiv: https://arxiv.org/abs/2604.00263 · HTML: https://mlsanity.com/paper/2604.00263/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 31 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [Domain-invariant feature learning in brain MR imaging for content-based image retrieval](https://mlsanity.com/paper/2501.01326/) (2025)
- [Adaptive Frequency Domain Alignment Network for Medical image segmentation](https://mlsanity.com/paper/2512.16393/) (2025)
- [Multimodal Learning for Scalable Representation of High-Dimensional Medical Data](https://mlsanity.com/paper/2409.13115/) (2024)

## Built on by (most cited first, 1 of 1)

- [What Do Medical Vision-Language Models Learn in Radiology? Transfer, Alignment, and Source-Proxy Leakage Under Distribution Shift](https://mlsanity.com/paper/2608.25251/) (2026)

## Abstract (arXiv)

Cross-hospital failure in chest X-ray models is often attributed to domain shift, yet most work assumes invariance without measuring it. This paper studies how to measure site leakage directly and how that measurement changes conclusions about transfer methods. We study multi-site self-supervised learning (SSL) and feature-level adversarial site confusion for cross-hospital transfer. We pretrain a ResNet-18 on NIH and CheXpert without pathology labels. We then freeze the encoder and train a linear pneumonia classifier on NIH only, evaluating transfer to RSNA. We quantify site leakage using a post hoc linear probe that predicts acquisition site from frozen backbone features f and projection features z. Across 3 random seeds, multi-site SSL improves RSNA AUC from 0.6736 ± 0.0148 (ImageNet initialization) to 0.7804 ± 0.0197. Adding adversarial site confusion on f reduces measured leakage but does not reliably improve AUC and increases variance. On f, site probe accuracy drops from 0.9890 ± 0.0021 (SSL-only) to 0.8504 ± 0.0051 (CanonicalF), where chance is 0.50. On z, probe accuracy drops from 0.8912 ± 0.0092 to 0.7810 ± 0.0250. These results show that measuring leakage changes how transfer methods should be interpreted: multi-site SSL drives transfer, while adversarial confusion exposes the limits of invariance assumptions.