mlsanityData to 14 Oct 2026

Privacy in Personalized AI Is a System Property, Not Just a Model Property

arXiv 2609.38289 ↗First posted 29 Sep 2026cs.CR cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In personalized AI applications, such as conversational assistants and recommender systems, users interact not with models in isolation but with broader systems that access, infer, and reuse user information across components and over time. While such use of user information is integral to personalization, it also raises important privacy questions. In this paper, we argue that individual model- or component-level analyses may not capture all privacy risks arising in such systems, motivating a system-level perspective on privacy. We distinguish and analyze four interconnected privacy-risk channels in personalized AI, and subsequently propose four requirements for system-level privacy evaluation, covering interaction trajectories, internal information flows, indirect leakage, and the privacy-utility trade-off. We argue for their systematic incorporation into privacy audits of personalized AI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

45 of 53 references.

References · 45

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  7. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  8. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  9. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  10. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  11. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  12. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
33 more
  1. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
  2. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  3. 2023Recommender AI Agent: Integrating Large Language Models for Interactive Recommendationscited by 131
  4. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  5. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  6. 2024Personalization of Large Language Models: A Surveycited by 104
  7. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 101
  8. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 82
  9. 2024Imprompter: Tricking LLM Agents into Improper Tool Usecited by 68
  10. 2023Privacy Issues in Large Language Models: A Surveycited by 60
  11. 2023Privacy Side Channels in Machine Learning Systemscited by 40
  12. 2024Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphscited by 38
  13. 2023Personalized Large Language Model Assistant with Evolving Conditional Memorycited by 25
  14. 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systemscited by 25
  15. 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Surveycited by 23
  16. 2019Privacy-Aware Recommendation with Private-Attribute Protection using Adversarial Learningcited by 21
  17. 2021A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Appscited by 21
  18. 2022Comprehensive Privacy Analysis on Federated Recommender System against Attribute Inference Attackscited by 21
  19. 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaborationcited by 20
  20. 2020Operationalizing the Legal Principle of Data Minimization for Personalizationcited by 19
  21. 2025Position: Privacy Is Not Just Memorization!cited by 17
  22. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutionscited by 15
  23. 2023Recent Developments in Recommender Systems: A Surveycited by 14
  24. 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
  25. 2025PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AIcited by 12
  26. 2022On the User Behavior Leakage from Recommender System Exposurecited by 7
  27. 2023A Scalable Framework for Automatic Playlist Continuation on Music Streaming Servicescited by 7
  28. 2026ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agentscited by 7
  29. 2023Track Mix Generation on Music Streaming Services using Transformerscited by 6
  30. 2026PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Saycited by 6
  31. 2026IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshucited by 2
  32. 2026A LINDDUN-based Privacy Threat Modeling Framework for GenAIcited by 1
  33. 2026Music Playlist Captioning at Scale with Large Language Modelscited by 1

Built on by

Not cited yet.