mlsanityData to 7 Oct 2026

Context-Aware Membership Inference Attacks against Pre-trained Large Language Models

arXiv 2409.13745 ↗First posted 11 Sep 2024cs.CL cs.AI cs.CRcited by 20

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs) aim at determining if a data point was part of the model's training set. Prior MIAs that are built for classification models fail at LLMs, due to ignoring the generative nature of LLMs across token sequences. In this paper, we present a novel attack on pre-trained LLMs that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 25 of its 55 references that are in the map. First posted on arXiv 11 Sep 2024.

Builds on

25 of 55 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)25 Jun
  2. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Models23 Jun
  3. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods23 Jun
  4. 2024LLM Dataset Inference: Did you train on my dataset?10 Jun
  5. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  6. 2023Low-Cost High-Power Membership Inference Attacks6 Dec
  7. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  8. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  9. 2021Membership Inference Attacks From First Principles7 Dec
  10. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov

Other references · 15

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  8. 2023Extracting Training Data from Diffusion Modelscited by 723
  9. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  10. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  11. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  12. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
3 more
  1. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  2. 2020Characterizing Structural Regularities of Labeled Data in Overparameterized Modelscited by 93
  3. 2017Choosing Between Methods of Combining p-valuescited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 13.

20 papers.

Sort:

By area:

  1. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  2. 2026Extracting books from production language modelscited by 16
  3. 2025Privacy Attacks on Image AutoRegressive Modelscited by 15
  4. 2025Mitigating Unintended Memorization with LoRA in Federated Learning for LLMscited by 6
  5. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  6. 2026LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systemscited by 6
  7. 2026Window-based Membership Inference Attacks Against Fine-tuned Large Language Modelscited by 5
  8. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  9. 2026Scalable next-scale autoregression for medical image generation across anatomical regionscited by 3
  10. 2025Membership and Dataset Inference Attacks on Large Audio Generative Modelscited by 2
  11. 2025NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptationcited by 1
  12. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
8 more
  1. 2026Private PoEtry: Private In-Context Learning via Product of Expertscited by 0
  2. 2026Boundary-targeted Membership Inference Attacks on Safety Classifierscited by 0
  3. 2026Natural Identifiers for Privacy and Data Audits in Large Language Modelscited by 0
  4. 2026Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queriescited by 0
  5. 2026Dataset Usage Inference without Shadow Models or Held-out Datacited by 0
  6. 2026Anchored or Drifting: What Recursive Self-Generation Reveals About Training Datacited by 0
  7. 2026JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Modelscited by 0
  8. 2026Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splittingcited by 0