On Protecting the Data Privacy of Large Language Models (LLMs): A Survey
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are complex artificial intelligence systems capable of understanding, generating and translating human language. They learn language patterns by analyzing large amounts of text data, allowing them to perform writing, conversation, summarizing and other language tasks. When LLMs process and generate large amounts of data, there is a risk of leaking sensitive information, which may threaten data privacy. This paper concentrates on elucidating the data privacy concerns associated with LLMs to foster a comprehensive understanding. Specifically, a thorough investigation is undertaken to delineate the spectrum of data privacy threats, encompassing both passive privacy leakage and active privacy attacks within LLMs. Subsequently, we conduct an assessment of the privacy protection mechanisms employed by LLMs at various stages, followed by a detailed examination of their efficacy and constraints. Finally, the discourse extends to delineate the challenges encountered and outline prospective directions for advancement in the realm of LLM privacy protection.
Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Mar 2024.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2024: 25; 2025: 55; 2026: 11.
91 papers.
Sort:
By area:
- 2025Small Language Models are the Future of Agentic AIcited by 254
- 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
- 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
- 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
- 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
- 2024A Survey of Useful LLM Evaluationcited by 28
- 2025A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcarecited by 27
- 2024The Future of AI-Driven Software Engineeringcited by 23
- 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
- 2024Multi-Head RAG: Solving Multi-Aspect Problems with LLMscited by 18
38 more
- 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 18
- 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protectioncited by 16
- 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
- 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystemscited by 15
- 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directionscited by 14
- 2025Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Modelscited by 14
- 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 13
- 2025Towards User-level Private Reinforcement Learning with Human Feedbackcited by 13
- 2024Efficient Prompting for LLM-based Generative Internet of Thingscited by 12
- 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Datacited by 10
- 2025SafeLawBench: Towards Safe Alignment of Large Language Modelscited by 9
- 2025From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graphcited by 9
- 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
- 2024Risks, Causes, and Mitigations of Widespread Deployments of Large Language Models (LLMs): A Surveycited by 8
- 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
- 2025On the Impact of Noise in Differentially Private Text Rewritingcited by 8
- 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
- 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysiscited by 7
- 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Callingcited by 7
- 2025LLM Meeting Decision Trees on Tabular Datacited by 7
- 2025Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Expertscited by 7
- 2024Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systemscited by 6
- 2024PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMscited by 6
- 2024Power-Softmax: Towards Secure LLM Inference over Encrypted Datacited by 6
- 2025AILS-NTUA at SemEval-2025 Task 4: Parameter-Efficient Unlearning for Large Language Models using Data Chunkingcited by 6
- 2025DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Surveycited by 6
- 2025LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliancecited by 6
- 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMscited by 6
- 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
- 2024Exactly Minimax-Optimal Locally Differentially Private Samplingcited by 5
- 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directionscited by 5
- 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigationcited by 5
- 2025A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Modelscited by 5
- 2025Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewritingcited by 5
- 2025Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysiscited by 5
- 2025Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implicationscited by 5
- 2025Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistantscited by 4
50 most cited of 91 All 91 (JSON)
- 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Models31 Aug 2026
- 2026PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement3 Jul 2026
- 2026Towards Large Model Feature Coding20 May 2026
- 2026Say Something Else: Rethinking Contextual Privacy as Information Sufficiency7 Apr 2026
- 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs1 Apr 2026
- 2026TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models25 Mar 2026
- 2026Differential Privacy in Generative AI Agents: Analysis and Optimal Tradeoffs18 Mar 2026
- 2026STAMP: Selective Task-Aware Mechanism for Text Privacy12 Mar 2026
- 2026FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards31 Jan 2026
- 2026Malicious Repurposing of Open Science Artefacts by Using Large Language Models26 Jan 2026
- 2026Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict7 Jan 2026
- 2025Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic25 Nov 2025
38 more
- 2025SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces10 Nov 2025
- 2025Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks1 Nov 2025
- 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMs31 Oct 2025
- 2025From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph22 Oct 2025
- 2025PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Models16 Oct 2025
- 2025Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMs16 Oct 2025
- 2025Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Technique7 Oct 2025
- 2025Towards Human-Centered RegTech: Unpacking Professionals' Strategies and Needs for Using LLMs Safely2 Oct 2025
- 2025"We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe1 Oct 2025
- 2025Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications10 Sep 2025
- 2025Efficient Decoding Methods for Language Models on Encrypted Data10 Sep 2025
- 2025A Survey: Towards Privacy and Security in Mobile Large Language Models2 Sep 2025
- 2025Comparative Analysis of Large Language Models for the Machine-Assisted Resolution of User Intentions29 Aug 2025
- 2025On the Security and Privacy of Federated Learning: A Survey with Attacks, Defenses, Frameworks, Applications, and Future Directions19 Aug 2025
- 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data16 Aug 2025
- 2025Data Security in Large Language Models: Risks, Defense, and Directions4 Aug 2025
- 2025Semantic Encryption: Secure and Effective Interaction with Cloud-based Large Language Models via Semantic Transformation3 Aug 2025
- 2025Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts1 Aug 2025
- 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystems5 Jul 2025
- 2025A Survey on Model Extraction Attacks and Defenses for Large Language Models26 Jun 2025
- 2025Towards Provable (In)Secure Model Weight Release Schemes23 Jun 2025
- 2025StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Framework17 Jun 2025
- 2025Collaborative Editable Model17 Jun 2025
- 2025SUA: Stealthy Multimodal Large Language Model Unlearning Attack10 Jun 2025
- 2025SafeLawBench: Towards Safe Alignment of Large Language Models7 Jun 2025
- 2025Small Language Models are the Future of Agentic AI2 Jun 2025
- 2025LLM Meeting Decision Trees on Tabular Data23 May 2025
- 2025LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliance22 May 2025
- 2025OpenEthics: A Comprehensive Ethical Evaluation of Open-Source Generative Large Language Models21 May 2025
- 2025UniErase: Towards Balanced and Precise Unlearning in Language Models21 May 2025
- 2025Private Transformer Inference in MLaaS: A Survey15 May 2025
- 2025Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis5 May 2025
- 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures2 May 2025
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage28 Apr 2025
- 2025Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Models27 Apr 2025
- 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling27 Apr 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025What Makes an Evaluation Useful? Common Pitfalls and Best Practices30 Mar 2025
50 newest of 91 All 91 (JSON)
- 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
- 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directionscited by 14
- 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
- 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
- 2025AILS-NTUA at SemEval-2025 Task 4: Parameter-Efficient Unlearning for Large Language Models using Data Chunkingcited by 6
- 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMscited by 6
- 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigationcited by 5
- 2025UniErase: Towards Balanced and Precise Unlearning in Language Modelscited by 4
- 2025Quantifying the Privacy Implications of High-Fidelity Synthetic Network Trafficcited by 3
- 2025PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Modelscited by 2
- 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
- 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMscited by 0
12 most cited of 13 All 13 (JSON)
- 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs1 Apr 2026
- 2025Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic25 Nov 2025
- 2025Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks1 Nov 2025
- 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMs31 Oct 2025
- 2025PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Models16 Oct 2025
- 2025Data Security in Large Language Models: Risks, Defense, and Directions4 Aug 2025
- 2025UniErase: Towards Balanced and Precise Unlearning in Language Models21 May 2025
- 2025AILS-NTUA at SemEval-2025 Task 4: Parameter-Efficient Unlearning for Large Language Models using Data Chunking4 Mar 2025
- 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation30 Jan 2025
- 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions21 Dec 2024
- 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states29 Nov 2024
- 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct 2024
12 newest of 13 All 13 (JSON)
- 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
- 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
- 2024A Survey of Useful LLM Evaluationcited by 28
- 2025A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcarecited by 27
- 2024The Future of AI-Driven Software Engineeringcited by 23
- 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
- 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 18
- 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
- 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystemscited by 15
- 2025Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Modelscited by 14
- 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 13
12 most cited of 62 All 62 (JSON)
- 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Models31 Aug 2026
- 2026Towards Large Model Feature Coding20 May 2026
- 2026Say Something Else: Rethinking Contextual Privacy as Information Sufficiency7 Apr 2026
- 2026TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models25 Mar 2026
- 2026STAMP: Selective Task-Aware Mechanism for Text Privacy12 Mar 2026
- 2026Malicious Repurposing of Open Science Artefacts by Using Large Language Models26 Jan 2026
- 2026Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict7 Jan 2026
- 2025Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMs16 Oct 2025
- 2025Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Technique7 Oct 2025
- 2025Towards Human-Centered RegTech: Unpacking Professionals' Strategies and Needs for Using LLMs Safely2 Oct 2025
- 2025"We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe1 Oct 2025
- 2025Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications10 Sep 2025
12 newest of 62 All 62 (JSON)