# Quantifying privacy risks in foundation model representations of brain MRI

arXiv: https://arxiv.org/abs/2610.20494 · HTML: https://mlsanity.com/paper/2610.20494/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Builds on (most cited first, 14 of 14)

- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images](https://mlsanity.com/paper/2310.15161/) (2023)
- [Overlearning Reveals Sensitive Attributes](https://mlsanity.com/paper/1905.11742/) (2019)
- [Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification](https://mlsanity.com/paper/1701.06643/) (2017)
- [BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation](https://mlsanity.com/paper/2406.10395/) (2024)
- [Anatomical Foundation Models for Brain MRIs](https://mlsanity.com/paper/2408.07079/) (2024)
- [General Framework to Evaluate Unlinkability in Biometric Template Protection Systems](https://mlsanity.com/paper/2311.04633/) (2023)
- [Building a General SimCLR Self-Supervised Foundation Model Across Neurological Diseases to Advance 3D Brain MRI Diagnoses](https://mlsanity.com/paper/2509.10620/) (2025)
- [Refacing: reconstructing anonymized facial features using GANs](https://mlsanity.com/paper/1810.06455/) (2018)
- [Towards Generalisable Foundation Models for Brain MRI](https://mlsanity.com/paper/2510.23415/) (2025)
- [A Modality-agnostic Multi-task Foundation Model for Human Brain Imaging](https://mlsanity.com/paper/2509.00549/) (2025)
- [DeepBrainPrint: A Novel Contrastive Framework for Brain MRI Re-Identification](https://mlsanity.com/paper/2302.13057/) (2023)
- [Advancing Brainwave-Based Biometrics: A Large-Scale, Multi-Session Evaluation](https://mlsanity.com/paper/2501.17866/) (2025)
- [Protecting patient privacy in clinical foundation models: Technical and legal perspectives](https://mlsanity.com/paper/2608.07705/) (2026)

## Abstract (arXiv)

Pretrained representations are increasingly used to share and reuse information from medical images without exposing the original scans, but it is unclear how much patient-specific information these representations retain. We investigate this question for brain magnetic resonance imaging (MRI) by evaluating five frozen pretrained encoders spanning different architectures, pretraining objectives, and training domains on two longitudinal T1-weighted datasets, OASIS-2 and MIRIAD. Under an attacker model with access to released embeddings, the corresponding public encoder, and auxiliary MRI scans, we evaluate patient linkage and sensitive-attribute inference. Across the evaluated representations, we observe substantial subject-specific information: rank-1 patient linkage reaches up to 99% on MIRIAD and exceeds 90% on OASIS-2 for the highest-risk representation, despite the absence of an identification objective during encoder training. We further show that the released embeddings retain information about a demographic attribute, with recorded-sex prediction reaching balanced accuracy of 90.9% on OASIS-2 and 95.5% on MIRIAD. Overall, our results demonstrate that replacing raw brain MRI with learned embeddings does not, by itself, eliminate patient-linkage or attribute-inference risk, and is therefore insufficient on its own to establish anonymity.