mlsanityData to 13 Oct 2026

A Survey on Privacy Risks and Protection in Large Language Models

arXiv 2505.01976 ↗First posted 4 May 2025cs.CRcited by 20

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Although Large Language Models (LLMs) have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. First, we analyze privacy leakage and attacks in LLMs, focusing on how these models unintentionally expose sensitive information through techniques such as model inversion, training data extraction, and membership inference. We investigate the mechanisms of privacy leakage, including the unauthorized extraction of training data and the potential exploitation of these vulnerabilities by malicious actors. Next, we review existing privacy protection against such risks, such as inference detection, federated learning, backdoor mitigation, and confidential computing, and assess their effectiveness in preventing privacy leakage. Furthermore, we highlight key practical challenges and propose future research directions to develop secure and privacy-preserving LLMs, emphasizing privacy risk assessment, secure knowledge transfer between models, and interdisciplinary frameworks for privacy governance. Ultimately, this survey aims to establish a roadmap for addressing escalating privacy challenges in the LLMs domain.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 72 of its 116 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

72 of 116 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  2. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  3. 2021On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models12 Mar

Other references · 69

  1. 2023GPT-4 Technical Reportcited by 18,278
  2. 2022Training language models to follow instructions with human feedbackcited by 15,856
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,999
  6. 2019Advances and Open Problems in Federated Learningcited by 3,040
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,586
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  9. 2020Extracting Training Data from Large Language Modelscited by 2,181
  10. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  11. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 579
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 398
38 more
  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  3. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 284
  4. 2023A Survey on Model Compression for Large Language Modelscited by 268
  5. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 246
  6. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 223
  7. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 211
  8. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 185
  9. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 173
  10. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 156
  11. 2023Large language models can accurately predict searcher preferencescited by 130
  12. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  13. 2020Data-Free Model Extractioncited by 124
  14. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 121
  15. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 112
  16. 2024BadEdit: Backdooring large language models by model editingcited by 96
  17. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 94
  18. 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 91
  19. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  20. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 82
  21. 2023Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimizationcited by 64
  22. 2023Privacy Issues in Large Language Models: A Surveycited by 59
  23. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 56
  24. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
  25. 2023ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relationscited by 40
  26. 2024Prompt Stealing Attacks Against Large Language Modelscited by 40
  27. 2022Fine-Tuning Is All You Need to Mitigate Backdoor Attackscited by 37
  28. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 37
  29. 2024GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theorycited by 36
  30. 2022Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformerscited by 31
  31. 2023Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibilitycited by 30
  32. 2023LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inferencecited by 27
  33. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Modelscited by 27
  34. 2023FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Modelscited by 26
  35. 2024Differentially Private Next-Token Prediction of Large Language Modelscited by 22
  36. 2024Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Modelscited by 17
  37. 2022Machine Learning with Confidential Computing: A Systematization of Knowledgecited by 15
  38. 2023PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learnerscited by 14

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 7; 2026: 13.

20 papers.

Sort:

By area:

  1. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approachescited by 20
  2. 2025A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Servicescited by 16
  3. 2025From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Modelscited by 3
  4. 2025Privacy Challenges and Solutions in Retrieval-Augmented Generation-Enhanced LLMs for Healthcare Chatbots: A Review of Applications, Risks, and Future Directionscited by 3
  5. 2026Simulating Complex Multi-Turn Tool Calling Interactions in Stateless Execution Environmentscited by 3
  6. 2026Auditing Training Data in Domain-adapted LLMs: LoRA-MINTcited by 3
  7. 2025An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEscited by 2
  8. 2026PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelinescited by 2
  9. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
  10. 2025EPT Benchmark: Evaluation of Persian Trustworthiness in Large Language Modelscited by 0
  11. 2026PriMod4AI: Lifecycle-Aware Privacy Threat Modeling for AI Systems using LLMcited by 0
  12. 2026A Privacy by Design Framework for Large Language Model-Based Applications for Childrencited by 0
8 more
  1. 2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuningcited by 0
  2. 2026Synthesizing real-world distributions from high-dimensional Gaussian Noise with Fully Connected Neural Networkcited by 0
  3. 2026SLM Finetuning for Natural Language to Domain Specific Code Generation in Productioncited by 0
  4. 2026On the Privacy of LLMs: An Ablation Studycited by 0
  5. 2026Benchmarking Local Language Models for Social Robots using Edge Devicescited by 0
  6. 2026KadiAssistant: A conversational AI Agent for information retrieval in Kadi4Matcited by 0
  7. 2026STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPUcited by 0
  8. 2026Implicit Reasoning Steering via Concept Chainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  2. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
  3. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
  4. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
  5. 2025The Model's Language Matters: A Comparative Privacy Analysis of LLMs
  6. 2024Analysis of Privacy Leakage in Federated Large Language Models
  7. 2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models
  8. 2025Privacy Auditing of Large Language Models