mlsanityData to 14 Oct 2026

Beyond Memorization: Violating Privacy Via Inference with Large Language Models

arXiv 2310.07298 ↗First posted 11 Oct 2023cs.AI cs.LGcited by 175

Not on the map. Why?

Cited by 36 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Current privacy research on large language models (LLMs) primarily focuses on the issue of extracting memorized training data. At the same time, models' inference capabilities have increased drastically. This raises the key question of whether current LLMs could violate individuals' privacy by inferring personal attributes from text given at inference time. In this work, we present the first comprehensive study on the capabilities of pretrained LLMs to infer personal attributes from text. We construct a dataset consisting of real Reddit profiles, and show that current LLMs can infer a wide range of personal attributes (e.g., location, income, sex), achieving up to 85% top-1 and 95% top-3 accuracy at a fraction of the cost (100×) and time (240×) required by humans. As people increasingly interact with LLM-powered chatbots across all aspects of life, we also explore the emerging threat of privacy-invasive chatbots trying to extract personal information through seemingly benign questions. Finally, we show that common mitigations, i.e., text anonymization and model alignment, are currently ineffective at protecting user privacy against LLM inference. Our findings highlight that current LLMs can infer personal data at a previously unattainable scale. In the absence of working defenses, we advocate for a broader discussion around LLM privacy implications beyond memorization, striving for a wider privacy protection.

Key facts. Cited by 175 papers in the map (data through 14 Oct 2026). Builds on 12 of its 62 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

12 of 62 references.

References · 12

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  8. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  9. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  10. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  11. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  12. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 46; 2025: 58; 2026: 66.

175 papers.

Sort:

By area:

  1. 2024Titans: Learning to Memorize at Test Timecited by 328
  2. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  3. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  4. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  6. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  7. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  8. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 98
  9. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 82
  10. 2024Large Language Model Safety: A Holistic Surveycited by 59
  11. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 56
  12. 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 47
38 more
  1. 2023Can LLMs Follow Simple Rules?cited by 46
  2. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
  3. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 37
  4. 2024Private Attribute Inference from Images with Vision-Language Modelscited by 37
  5. 2024Offset Unlearning for Large Language Modelscited by 36
  6. 2024A Synthetic Dataset for Personal Attribute Inferencecited by 36
  7. 2024Protecting Your LLMs with Information Bottleneckcited by 29
  8. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 29
  9. 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agentcited by 29
  10. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  11. 2024COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Actcited by 24
  12. 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 23
  13. 2024Large Language Models are Advanced Anonymizerscited by 22
  14. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 22
  15. 2024Can Language Models Recognize Convincing Arguments?cited by 21
  16. 2025A Survey on Privacy Risks and Protection in Large Language Modelscited by 21
  17. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 19
  18. 2025Robustly Improving LLM Fairness in Realistic Settings via Interpretabilitycited by 18
  19. 2024Can Language Models Act as Knowledge Bases at Scale?cited by 17
  20. 2025Security of Internet of Agents: Attacks and Countermeasurescited by 17
  21. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  22. 2025Position: Privacy Is Not Just Memorization!cited by 17
  23. 2024An In-Depth Investigation of Data Collection in LLM App Ecosystemscited by 14
  24. 2026Large-scale online deanonymization with LLMscited by 14
  25. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  26. 2024Measuring memorization in RLHF for code completioncited by 13
  27. 2024Detecting Training Data of Large Language Models via Expectation Maximizationcited by 13
  28. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 13
  29. 2024Privacy-Aware Visual Language Modelscited by 12
  30. 2024Evaluating LLM-based Personal Information Extraction and Countermeasurescited by 12
  31. 2024Mitigating Memorization In Language Modelscited by 12
  32. 2025Evaluating Precise Geolocation Inference Capabilities of Vision Language Modelscited by 12
  33. 2024ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generationcited by 11
  34. 2025Privacy Reasoning in Ambiguous Contextscited by 11
  35. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigationcited by 11
  36. 2026DevOps-Gym: Benchmarking AI Agents in Software DevOps Cyclecited by 11
  37. 2025Agents Are All You Need for LLM Unlearningcited by 10
  38. 2025Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Modelscited by 10

50 most cited of 175 All 175 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
  2. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
  3. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  4. 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data
  5. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration
  6. 2024Understanding Privacy Risks of Embeddings Induced by Large Language Models
  7. 2025Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences
  8. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory