mlsanityData to 7 Oct 2026

PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action

arXiv 2409.00138 ↗First posted 29 Aug 2024cs.CL cs.AI cs.CRcited by 161

Not on the map. Why?

Cited by 105 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As language models (LMs) are widely utilized in personalized communication scenarios (e.g., sending emails, writing social media posts) and endowed with a certain level of agency, ensuring they act in accordance with the contextual privacy norms becomes increasingly critical. However, quantifying the privacy norm awareness of LMs and the emerging privacy risk in LM-mediated communication is challenging due to (1) the contextual and long-tailed nature of privacy-sensitive cases, and (2) the lack of evaluation approaches that capture realistic application scenarios. To address these challenges, we propose PrivacyLens, a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories, enabling multi-level evaluation of privacy leakage in LM agents' actions. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. Using this dataset, we reveal a discrepancy between LM performance in answering probing questions and their actual behavior when executing user instructions in an agent setup. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions. We also demonstrate the dynamic nature of PrivacyLens by extending each seed into multiple trajectories to red-team LM privacy leakage risk. Dataset and code are available at https://github.com/SALT-NLP/PrivacyLens.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). Builds on 36 of its 72 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

36 of 72 references.

References · 36

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Mistral 7Bcited by 3,065
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  12. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
24 more
  1. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  2. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  3. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  4. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  5. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  7. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  8. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  9. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  10. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  11. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  12. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  13. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 127
  14. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  15. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 91
  16. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  17. 2020The Turking Test: Can Language Models Understand Instructions?cited by 82
  18. 2024Can Language Models Solve Olympiad Programming?cited by 70
  19. 2023Testing Language Model Agents Safely in the Wildcited by 45
  20. 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
  21. 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 40
  22. 2023The Consensus Game: Language Model Generation via Equilibrium Searchcited by 35
  23. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  24. 2023ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systemscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 51; 2026: 103.

161 papers.

Sort:

By area:

  1. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 104
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  5. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 82
  6. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 79
  7. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 71
  8. 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaborationcited by 60
  9. 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injectionscited by 55
  10. 2024PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensemblescited by 54
  11. 2025UXAgent: An LLM Agent-Based Usability Testing Framework for Web Designcited by 39
  12. 2025Firewalls to Secure Dynamic LLM Agentic Networkscited by 34
38 more
  1. 2025PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliancecited by 34
  2. 2025Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agentscited by 33
  3. 2025LLM Agents Should Employ Security Principlescited by 28
  4. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  5. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  6. 2025SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraintscited by 21
  7. 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systemscited by 20
  8. 2025Searching for Privacy Risks in LLM Agents via Simulationcited by 19
  9. 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
  10. 20251-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoningcited by 16
  11. 2025Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversightcited by 15
  12. 2025Position: Privacy Is Not Just Memorization!cited by 15
  13. 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agentcited by 14
  14. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  15. 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversationscited by 14
  16. 2025EgoNormia: Benchmarking Physical Social Norm Understandingcited by 13
  17. 2025Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agentscited by 11
  18. 2025Privacy Reasoning in Ambiguous Contextscited by 11
  19. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigationcited by 11
  20. 2025UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agentscited by 10
  21. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 10
  22. 2026CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agentscited by 10
  23. 2026AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflowscited by 9
  24. 2026AI Agents May Always Fall for Prompt Injectionscited by 9
  25. 2025Web Fraud Attacks Against LLM-Driven Multi-Agent Systemscited by 8
  26. 2025Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directednesscited by 8
  27. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  28. 2026Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systemscited by 8
  29. 2026HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmarkcited by 8
  30. 2026Auditing Agent Harness Safetycited by 8
  31. 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 7
  32. 2026Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Usecited by 7
  33. 2026Learning Next Action Predictors from Human-Computer Interactioncited by 7
  34. 2026AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networkscited by 7
  35. 2026MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agentscited by 7
  36. 2024Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metriccited by 6
  37. 2024Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Modelscited by 6
  38. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6

50 most cited of 161 All 161 (JSON)