mlsanityData to 14 Oct 2026

SoK: Semantic Privacy in Large Language Models

arXiv 2506.23603 ↗First posted 30 Jun 2025cs.CR cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

As Large Language Models (LLMs) are increasingly deployed in sensitive domains, traditional data privacy measures prove inadequate for protecting information that is implicit, contextual, or inferable - what we define as semantic privacy. This Systematization of Knowledge (SoK) introduces a lifecycle-centric framework to analyze how semantic privacy risks emerge across input processing, pretraining, fine-tuning, and alignment stages of LLMs. We categorize key attack vectors and assess how current defenses, such as differential privacy, embedding encryption, edge computing, and unlearning, address these threats. Our analysis reveals critical gaps in semantic-level protection, especially against contextual inference and latent representation leakage. We conclude by outlining open challenges, including quantifying semantic leakage, protecting multimodal inputs, balancing de-identification with generation quality, and ensuring transparency in privacy enforcement. This work aims to inform future research on designing robust, semantically aware privacy-preserving techniques for LLMs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 17 of its 39 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

17 of 39 references.

References · 17

  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  2. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  3. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  4. 2023Large language models in healthcare and medical domain: A reviewcited by 127
  5. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  6. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
  7. 2023Privacy Issues in Large Language Models: A Surveycited by 60
  8. 2024Membership Inference Attacks Against In-Context Learningcited by 43
  9. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 42
  10. 2024Extracting Prompts by Inverting LLM Outputscited by 41
  11. 2024Strong and weak alignment of large language models with human valuescited by 27
  12. 2024Noisy Neighbors: Efficient membership inference attacks against LLMscited by 21
5 more
  1. 2024TexShape: Information Theoretic Sentence Embedding for Language Modelscited by 10
  2. 2023SentinelLMs: Encrypted Input Adaptation and Fine-tuning of Language Models for Private and Secure Inferencecited by 9
  3. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  4. 2025UniErase: Towards Balanced and Precise Unlearning in Language Modelscited by 5
  5. 2025Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attackscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 130
  2. 2025Prompt to Pwn: Automated Exploit Generation for Smart Contractscited by 4
  3. 2026PlanTwin: Privacy-Preserving Planning Abstractions for Cloud-Assisted LLM Agentscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Privacy Risks and Protection in Large Language Models
  2. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
  3. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  4. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
  5. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
  6. 2024SoK: Reducing the Vulnerability of Fine-tuned Language Models to Membership Inference Attacks
  7. 2025The Model's Language Matters: A Comparative Privacy Analysis of LLMs
  8. 2026LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication