# On the Privacy of LLMs: An Ablation Study

arXiv: https://arxiv.org/abs/2605.02255 · HTML: https://mlsanity.com/paper/2605.02255/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 4 May 2026.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 27 of 27)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [Counterfactual Memorization in Neural Language Models](https://mlsanity.com/paper/2112.12938/) (2021)
- [The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies](https://mlsanity.com/paper/2407.19354/) (2024)
- [BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models](https://mlsanity.com/paper/2401.12242/) (2024)
- [Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents](https://mlsanity.com/paper/2411.09523/) (2024)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models](https://mlsanity.com/paper/2408.12798/) (2024)
- [A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networks](https://mlsanity.com/paper/2308.14367/) (2023)
- [A Survey of Recent Backdoor Attacks and Defenses in Large Language Models](https://mlsanity.com/paper/2406.06852/) (2024)
- [Mask-based Membership Inference Attacks for Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20142/) (2024)
- [Security Concerns for Large Language Models: A Survey](https://mlsanity.com/paper/2505.18889/) (2025)
- [A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations](https://mlsanity.com/paper/2502.05224/) (2025)
- [Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.19234/) (2024)
- [Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2406.07973/) (2024)
- [A Survey on Privacy Risks and Protection in Large Language Models](https://mlsanity.com/paper/2505.01976/) (2025)
- [TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning](https://mlsanity.com/paper/2404.19597/) (2024)

## Abstract (arXiv)

Large language models (LLMs) face diverse privacy threats, yet existing studies typically evaluate individual attacks in isolation, offering limited guidance on which system factors drive privacy vulnerabilities. This paper presents a unified threat model and a systematic ablation study of four representative attacks: Membership Inference (MIA), Attribute Inference (AIA), Data Extraction (DEA), and Backdoor Attacks (BA). Our findings reveal attack-specific sensitivities: membership inference depends strongly on dataset characteristics and attack strategy, attribute inference is influenced by model and training characteristics, data extraction increases with model scale and training-data repetition, and backdoor attacks achieve consistently high success rates. Based on these findings, we derive practical recommendations for privacy-aware LLM deployment, including prioritizing membership leakage assessments in retrieval-augmented systems, minimizing repeated personal information in training data, monitoring attribute inference risks, and safeguarding fine-tuning pipelines against backdoors.