AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
With the rise of personalized, persistent LLM agent frameworks such as OpenClaw, human-centered agentic social networks in which teams of collaborative AI agents serve individual users in a social network across multiple domains are becoming a reality. This setting creates novel privacy challenges: agents must coordinate across domain boundaries, mediate between humans, and interact with other users' agents, all while protecting sensitive personal information. While prior work has evaluated multi-agent coordination and privacy preservation, the dynamics and privacy risks of human-centered agentic social networks remain unexplored. To this end, we introduce AgentSocialBench, the first benchmark to systematically evaluate privacy risk in this setting, comprising scenarios across seven categories spanning dyadic and multi-party interactions, grounded in realistic user profiles with hierarchical sensitivity labels and directed social graphs. Our experiments reveal that privacy in agentic social networks is fundamentally harder than in single-agent settings: (1) cross-domain and cross-user coordination creates persistent leakage pressure even when agents are explicitly instructed to protect information, (2) privacy instructions that teach agents how to abstract sensitive information paradoxically cause them to discuss it more (we call it abstraction paradox). These findings underscore that current LLM agents lack robust mechanisms for privacy preservation in human-centered agentic social networks, and that new approaches beyond prompt engineering are needed to make agent-mediated social coordination safe for real-world deployment.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 47 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
25 of 47 references.
Closest ancestors · 4
Same direction, newest first
- 2025Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs4 Dec
- 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation16 Oct
- 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
- 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
Other references · 21
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2024Qwen2 Technical Reportcited by 2,076
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
- 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
- 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
9 more
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
- 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 127
- 2023MindAgent: Emergent Gaming Interactioncited by 87
- 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 35
- 2026"Humans welcome to observe": A First Look at the Agent Social Network Moltbookcited by 25
- 2025CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMscited by 22
- 2026Does Socialization Emerge in AI Agent Society? A Case Study of Moltbookcited by 13
- 2026Collective Behavior of AI Agents: the Case of Moltbookcited by 12
- 2026MoltNet: Understanding Social Behavior of AI Agents in the Agent-Native MoltBookcited by 12
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversationscited by 2
- 2026SOTOPIA-TOM: Evaluating Privacy and Information Management in Multi-Agent Interaction with Theory of Mindcited by 1
- 2026"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agentscited by 1
- 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systemscited by 1
- 2026WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networkscited by 1
- 2026CoSec: Benchmarking Agent Security in Communitiescited by 0
- 2026EP-Mem: Elastic Privacy Memory for Social Relationship-Aware LLM Agentscited by 0
- 2026EP-Mem: Elastic Privacy Memory for Social Relationship-Aware LLM Agents28 Sep 2026
- 2026CoSec: Benchmarking Agent Security in Communities28 Sep 2026
- 2026WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks4 Aug 2026
- 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems6 Jul 2026
- 2026MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations22 Jun 2026
- 2026"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents30 May 2026
- 2026SOTOPIA-TOM: Evaluating Privacy and Information Management in Multi-Agent Interaction with Theory of Mind4 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
- 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agents
- 2026From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management
- 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations
- 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration
- 2025PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI
- 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaboration