mlsanityData to 14 Oct 2026

Membership Inference Attacks on Sequence Models

arXiv 2506.05126 ↗First posted 5 Jun 2025cs.CR cs.LGcited by 6

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Sequence models, such as Large Language Models (LLMs) and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that those tools' shortcomings are due to mismatched assumptions. Thus, we argue that effectively measuring privacy leakage in sequence models requires leveraging the correlations inherent in sequential generation. To illustrate this, we adapt a state-of-the-art membership inference attack to explicitly model within-sequence correlations, thereby demonstrating how a strong existing attack can be naturally extended to suit the structure of sequence models. Through a case study, we show that our adaptations consistently improve the effectiveness of memorization audits without introducing additional computational costs. Our work hence serves as an important stepping stone toward reliable memorization audits for large sequence models.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 36 of its 47 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

36 of 47 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data29 Sep
  2. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  3. 2023Low-Cost High-Power Membership Inference Attacks6 Dec
  4. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  5. 2021Membership Inference Attacks From First Principles7 Dec
  6. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  7. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov

Other references · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2023PaLM 2 Technical Reportcited by 1,162
  8. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  9. 2023Large Language Modelscited by 780
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  11. 2023Extracting Training Data from Diffusion Modelscited by 732
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
17 more
  1. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  3. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  4. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  5. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  6. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  7. 2022The Privacy Onion Effect: Memorization is Relativecited by 136
  8. 2023Privacy Auditing with One (1) Training Runcited by 123
  9. 2024Scalable Pre-training of Large Autoregressive Image Modelscited by 115
  10. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 112
  11. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 103
  12. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 101
  13. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 88
  14. 2024Evaluations of Machine Learning Privacy Defenses are Misleadingcited by 36
  15. 2024Memorization in Self-Supervised Learning Improves Downstream Generalizationcited by 18
  16. 2024Measuring Non-Adversarial Reproduction of Training Data in Large Language Modelscited by 14
  17. 2024Extracting Training Data from Document-Based VQA Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Modelscited by 4
  2. 2026Powerful Training-Free Membership Inference Against Autoregressive Language Modelscited by 1
  3. 2026Membership Inference Attack Against Music Diffusion Models via Generative Manifold Perturbationcited by 1
  4. 2025Privacy Risks in Time Series Forecasting: User- and Record-Level Membership Inferencecited by 0
  5. 2026Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimatorscited by 0
  6. 2026Boundary-targeted Membership Inference Attacks on Safety Classifierscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PerProb: Indirectly Evaluating Memorization in Large Language Models ↗
  2. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks
  3. 2025(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs
  4. 2026Window-based Membership Inference Attacks Against Fine-tuned Large Language Models
  5. 2024Undesirable Memorization in Large Language Models: A Survey
  6. 2025Privacy Auditing of Large Language Models
  7. 2024Order of Magnitude Speedups for LLM Membership Inference
  8. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Models