# How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors? arXiv: https://arxiv.org/abs/2602.18464 ยท HTML: https://mlsanity.com/paper/2602.18464/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 34 of its 100 references that are in the map. - First posted on arXiv 6 Feb 2026. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) ## Builds on (most cited first, 30 of 34) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023) - [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023) - [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022) - [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023) - [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022) - [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025) - [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024) - [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022) - [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024) - [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024) - [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024) - [Simulating Classroom Education with LLM-Empowered Agents](https://mlsanity.com/paper/2406.19226/) (2024) - [Epidemic Modeling with Generative Agents](https://mlsanity.com/paper/2307.04986/) (2023) - [Finetuning LLMs for Human Behavior Prediction in Social Science Experiments](https://mlsanity.com/paper/2509.05830/) (2025) - [OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents](https://mlsanity.com/paper/2410.21286/) (2024) - [Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models](https://mlsanity.com/paper/2501.17420/) (2025) - [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024) - [MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations](https://mlsanity.com/paper/2504.07830/) (2025) - [AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms](https://mlsanity.com/paper/2306.03280/) (2023) - [The Stepwise Deception: Simulating the Evolution from True News to Fake News with LLM Agents](https://mlsanity.com/paper/2410.19064/) (2024) - [(Un)informed Consent: Studying GDPR Consent Notices in the Field](https://mlsanity.com/paper/1909.02638/) (2019) - [LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra](https://mlsanity.com/paper/2507.15815/) (2025) - [Searching for Privacy Risks in LLM Agents via Simulation](https://mlsanity.com/paper/2508.10880/) (2025) - [ExploreGen: Large Language Models for Envisioning the Uses and Risks of AI Technologies](https://mlsanity.com/paper/2407.12454/) (2024) - [Co-designing an AI Impact Assessment Report Template with AI Practitioners and AI Compliance Experts](https://mlsanity.com/paper/2407.17374/) (2024) - [SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation](https://mlsanity.com/paper/2504.16122/) (2025) - [Improving Language Model Personas via Rationalization with Psychological Scaffolds](https://mlsanity.com/paper/2504.17993/) (2025) - [SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent](https://mlsanity.com/paper/2410.14152/) (2024) - [SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator](https://mlsanity.com/paper/2505.17735/) (2025) ## Built on by (most cited first, 3 of 3) - [An AI Agent Execution Environment to Safeguard User Data](https://mlsanity.com/paper/2604.19657/) (2026) - [WhatIf: Interactive Exploration of LLM-Powered Social Simulations for Policy Reasoning](https://mlsanity.com/paper/2604.17615/) (2026) - [PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior](https://mlsanity.com/paper/2605.12147/) (2026) ## Abstract (arXiv) A growing body of research assumes that large language model (LLM) agents can serve as proxies for how people form attitudes toward and behave in response to security and privacy (S&P) threats. If correct, these simulations could offer a scalable way to forecast S&P risks in products prior to deployment. We interrogate this assumption using SP-ABCBench, a new benchmark of 30 tests derived from validated S&P human-subject studies, which measures alignment between simulations and human-subjects studies on a 0-100 ascending scale, where higher scores indicate better alignment across three dimensions: Attitude, Behavior, and Coherence. Evaluating twelve LLMs, four persona construction strategies, and two prompting methods, we found that there remains substantial room for improvement: all models score between 50 and 64 on average. Newer, bigger, and smarter models do not reliably do better and sometimes do worse. Some simulation configurations, however, do yield high alignment: e.g., with scores above 95 for some behavior tests when agents are prompted to apply bounded rationality and weigh privacy costs against perceived benefits. We release SP-ABCBench to enable reproducible evaluation as methods improve.