mlsanityData to 7 Oct 2026

On the Privacy of LLMs: An Ablation Study

arXiv 2605.02255 ↗First posted 4 May 2026cs.CR cs.AIcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in interactive and retrieval-augmented settings, raising significant privacy concerns. While attacks such as Membership Inference (MIA), Attribute Inference (AIA), Data Extraction (DEA), and Backdoor Attacks (BA) have been studied, they are typically analyzed in isolation, leaving a gap in understanding their behavior under common system factors. In this paper, we introduce a unified threat model and notation, reproduce a representative set of privacy attacks, and conduct a structured ablation study to evaluate the impact of key factors such as model architecture, scale, dataset characteristics, and retrieval configuration. Our analysis reveals clear differences across attack types. Membership inference attacks, particularly mask-based variants, exhibit strong and reliable signals, while backdoor attacks achieve consistently high success rates due to their trigger-based nature. In contrast, attribute inference and data extraction attacks remain more challenging, resulting in lower accuracy, yet they pose significant risks as they target sensitive personal information. Overall, these results highlight that privacy risks in LLM systems are highly context-dependent and driven by design choices, emphasizing the need for holistic evaluation and informed deployment practices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 36 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

27 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generation26 Oct
  2. 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation27 Jun

Other references · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  5. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  10. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  11. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  12. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
13 more
  1. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  2. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  3. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  4. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  5. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  6. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
  7. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 38
  8. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  9. 2025Security Concerns for Large Language Models: A Surveycited by 26
  10. 2025A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluationscited by 25
  11. 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Surveycited by 20
  12. 2025A Survey on Privacy Risks and Protection in Large Language Modelscited by 20
  13. 2024TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuningcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
  2. 2025Membership Inference Attacks on Large-Scale Models: A Survey
  3. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
  4. 2025Ensemble Privacy Defense for Knowledge-Intensive LLMs against Membership Inference Attacks
  5. 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs
  6. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text
  7. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment
  8. 2024Understanding Privacy Risks of Embeddings Induced by Large Language Models