mlsanityData to 7 Oct 2026

Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration

arXiv 2311.06062 ↗First posted 10 Nov 2023cs.CL cs.CR cs.LGcited by 90

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership Inference Attacks (MIA) aim to infer whether a target data record has been utilized for model training or not. Existing MIAs designed for large language models (LLMs) can be bifurcated into two types: reference-free and reference-based attacks. Although reference-based attacks appear promising performance by calibrating the probability measured on the target model with reference models, this illusion of privacy risk heavily depends on a reference dataset that closely resembles the training set. Both two types of attacks are predicated on the hypothesis that training records consistently maintain a higher probability of being sampled. However, this hypothesis heavily relies on the overfitting of target models, which will be mitigated by multiple regularization methods and the generalization of LLMs. Thus, these reasons lead to high false-positive rates of MIAs in practical scenarios. We propose a Membership Inference Attack based on Self-calibrated Probabilistic Variation (SPV-MIA). Specifically, we introduce a self-prompt approach, which constructs the dataset to fine-tune the reference model by prompting the target LLM itself. In this manner, the adversary can collect a dataset with a similar distribution from public APIs. Furthermore, we introduce probabilistic variation, a more reliable membership signal based on LLM memorization rather than overfitting, from which we rediscover the neighbour attack with theoretical grounding. Comprehensive evaluation conducted on three datasets and four exemplary LLMs shows that SPV-MIA raises the AUC of MIAs from 0.7 to a significantly high level of 0.9. Our code and dataset are available at: https://github.com/tsinghua-fib-lab/NeurIPS2024_SPV-MIA

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 43 of its 73 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

43 of 73 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models23 Aug
  2. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  3. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?2 Feb
  4. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  5. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  6. 2021Membership Inference Attacks From First Principles7 Dec
  7. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  8. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
  9. 2021Membership Inference Attacks Against Recommender Systems16 Sep
  10. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar

Other references · 33

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2015Teaching Machines to Read and Comprehendcited by 1,039
  11. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  12. 2022Quantifying Memorization Across Neural Language Modelscited by 755
21 more
  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  2. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  3. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  5. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  6. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  7. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  8. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  9. 2020Label-Only Membership Inference Attackscited by 371
  10. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  11. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  12. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  13. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  14. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 183
  15. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  16. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  17. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  18. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  19. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  20. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 51
  21. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 27; 2025: 35; 2026: 27.

90 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  3. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  4. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  5. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  6. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  7. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  8. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoodscited by 61
  9. 2024Dated Data: Tracing Knowledge Cutoffs in Large Language Modelscited by 48
  10. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  11. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 38
  12. 2024Membership Inference Attacks Against In-Context Learningcited by 38
38 more
  1. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 36
  2. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Modelscited by 34
  3. 2024Membership Inference Attacks against Large Vision-Language Modelscited by 31
  4. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignmentcited by 24
  5. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Modelscited by 21
  6. 2024Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decodingcited by 20
  7. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directionscited by 14
  8. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detectorcited by 13
  9. 2024A Comprehensive Survey of Contamination Detection Methods in Large Language Modelscited by 12
  10. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 10
  11. 2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Taskscited by 10
  12. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  13. 2024A Middle Path for On-Premises LLM Deployment: Preserving Privacy Without Sacrificing Model Confidentialitycited by 8
  14. 2025Unlocking Post-hoc Dataset Inference with Synthetic Datacited by 7
  15. 2024A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finishcited by 6
  16. 2025Auditing Data Membership in Reinforcement Learning With Verifiable Rewardscited by 6
  17. 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMscited by 6
  18. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  19. 2025Revisiting Privacy, Utility, and Efficiency Trade-offs when Fine-Tuning Large Language Modelscited by 5
  20. 2026Window-based Membership Inference Attacks Against Fine-tuned Large Language Modelscited by 5
  21. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  22. 2024Extracting Memorized Training Data via Decompositioncited by 4
  23. 2025Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks?cited by 4
  24. 2025Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Modelscited by 4
  25. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  26. 2024Privacy Evaluation Benchmarks for NLP Modelscited by 3
  27. 2025Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacycited by 3
  28. 2025Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Modelscited by 3
  29. 2025Gradient Inversion Attacks on Parameter-Efficient Fine-Tuningcited by 3
  30. 2025Benchmarking LLM Privacy Recognition for Social Robot Decision Makingcited by 3
  31. 2025Membership and Memorization in LLM Knowledge Distillationcited by 3
  32. 2025Enterprise AI Must Enforce Participant-Aware Access Controlcited by 3
  33. 2025Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probingcited by 3
  34. 2025In-Context Probing for Membership Inference in Fine-Tuned Language Modelscited by 3
  35. 2024Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language modelscited by 2
  36. 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
  37. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
  38. 2025Automatic Calibration for Membership Inference Attack on Large Language Modelscited by 2

50 most cited of 90 All 90 (JSON)