mlsanityData to 14 Oct 2026

Leak It: Per-Document Extraction Beyond Aggregate Membership Inference

arXiv 2608.00144 ↗First posted 31 Jul 2026cs.LG cs.CL cs.CRcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership inference (MIA) on language models is usually summarised by aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines can separate members from non-members using surface text alone. Building on probabilistic discoverable extraction, we study black-box training-data leakage using N samples from p_theta(. | x), placing mean overlap, extreme-value overlap, and self-concentration on a common functional-estimation footing. On WikiMIA, a blind bag-of-words classifier reaches AUC 0.97 (TPR 0.90 at 5% FPR) while sampling adds nothing. On an IID Pile split (MIMIR), neither self-concentration nor gold-continuation recovery significantly exceeds a blind baseline in aggregate. Aggregate metrics hide the real harm: sampling verbatim-extracts training data for a tail of documents no blind attack can reach. On Pythia-6.9B, 16.6% of 500 Pile documents bearing a real identifier (83 documents; 21.3% of those bearing an email address) have that identifier reproduced and not reproduced under a mismatched-prefix control. Each leak is attributable to that document rather than a globally common string. This per-document disclosure is invisible to aggregate AUC. Risk is uneven: identifier leakage is about 3x stronger in code than prose, though prose remains positive and grows with capacity (4.0% to 12.1% from 410M to 6.9B); recovery of arbitrary held-out continuations is essentially confined to code (+0.44 member gap on GitHub vs at most +0.014 on prose). Temperature and nucleus sampling have minor effect, a 16-token prefix suffices, and the sample-budget relationship corroborates prior probabilistic-extraction results. We detect no reduction from deduplication. Privacy audits should report per-document extraction, not only aggregate membership, and motivate differential privacy as the mitigation. We release leakit, a black-box tool implementing this probe and its control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 22 of its 26 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

22 of 26 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026AttenMIA: LLM Membership Inference Attack through Attention Signals26 Jan
  2. 2025Exploring the limits of strong membership inference attacks on large language models24 May
  3. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models26 Feb
  4. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Models23 Jun
  5. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods23 Jun
  6. 2024LLM Dataset Inference: Did you train on my dataset?10 Jun
  7. 2024Sampling-based Pseudo-Likelihood for Membership Inference Attacks17 Apr
  8. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  9. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  10. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  11. 2023Analyzing Leakage of Personally Identifiable Information in Language Models1 Feb

Other references · 11

  1. 2020Extracting Training Data from Large Language Modelscited by 2,189
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  5. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  6. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  7. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  8. 2023Emergent and Predictable Memorization in Large Language Modelscited by 148
  9. 2024Measuring memorization in language models via probabilistic extractioncited by 41
  10. 2025Extracting memorized pieces of (copyrighted) books from open-weight language modelscited by 30
  11. 2023MoPe: Model Perturbation-based Privacy Attacks on Language Modelscited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Detecting Data Contamination in Large Language Models
  2. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
  3. 2025On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Models
  4. 2023Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks
  5. 2024A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Models
  6. 2025(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs
  7. 2024Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction
  8. 2024Detecting Training Data of Large Language Models via Expectation Maximization