# Memorisation bias in medical AI

arXiv: https://arxiv.org/abs/2609.17223 · HTML: https://mlsanity.com/paper/2609.17223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Towards Accurate Differential Diagnosis with Large Language Models](https://mlsanity.com/paper/2312.00164/) (2023)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [Benchmarking emergency department triage prediction models with machine learning and large public electronic health records](https://mlsanity.com/paper/2111.11017/) (2021)
- [Scaling Laws for Differentially Private Language Models](https://mlsanity.com/paper/2501.18914/) (2025)
- [Exploring the limits of strong membership inference attacks on large language models](https://mlsanity.com/paper/2505.18773/) (2025)
- [Memorization in Self-Supervised Learning Improves Downstream Generalization](https://mlsanity.com/paper/2401.12233/) (2024)
- [hyppo: A Multivariate Hypothesis Testing Python Package](https://mlsanity.com/paper/1907.02088/) (2019)
- [The Chi-Square Test of Distance Correlation](https://mlsanity.com/paper/1912.12150/) (2019)
- [An Investigation of Memorization Risk in Healthcare Foundation Models](https://mlsanity.com/paper/2510.12950/) (2025)

## Abstract (arXiv)

Medical AI models hold immense potential to improve patient outcomes, but they are also known to unintentionally memorise individual records from their training datasets. While such memorisation has been linked to targeted privacy attacks, its consequences for clinical deployment, where patients may be assessed by a model that saw their historical data during training, remain poorly understood. Here we show that predictions on a patient's unseen future data can change significantly if a model observed that same patient's anonymised historical data during training, a phenomenon we term "memorisation bias". We demonstrate that this bias exists across diverse data modalities and model architectures, and over prolonged time spans: in some cases, memorisation bias persists on future records acquired decades after the historical records used for training. Moreover, in simulated prospective deployment, memorisation bias has asymmetric effects on the diagnostic accuracy of returning data contributors. When a patient returned with a de novo condition absent from their historical records in the training dataset, diagnostic sensitivity decreased significantly compared to an otherwise identical model not trained on their historical data. Conversely, when their health state was unchanged, both sensitivity and specificity were significantly inflated. Our findings reveal a previously uncharacterised risk in medical AI that arises when a model is deployed on patients who contributed to its training data. This exposes a shortcoming of current model development practice: the de-identification measures designed to protect patients' privacy make it difficult to identify returning contributors and exclude them from the AI-assisted interpretation of their own future data. Mitigating memorisation risks may thus require changes to current model training and deployment protocols.