# Do LLM Agents Exhibit Social Behavior?

arXiv: https://arxiv.org/abs/2312.15198 · HTML: https://mlsanity.com/paper/2312.15198/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 23 Dec 2023.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 19 of 19)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming](https://mlsanity.com/paper/2310.06500/) (2023)
- [Susceptibility to Influence of Large Language Models](https://mlsanity.com/paper/2303.06074/) (2023)
- [Gift Contagion in Online Groups: Evidence From Virtual Red Packets](https://mlsanity.com/paper/1906.09698/) (2019)
- [Interpretable Stochastic Block Influence Model: measuring social influence among homophilous communities](https://mlsanity.com/paper/2006.01028/) (2020)

## Built on by (most cited first, 30 of 53)

- [Game-theoretic LLM: Agent Workflow for Negotiation Games](https://mlsanity.com/paper/2411.05990/) (2024)
- [SocialBench: Sociality Evaluation of Role-Playing Conversational Agents](https://mlsanity.com/paper/2403.13679/) (2024)
- [Y Social: an LLM-powered Social Media Digital Twin](https://mlsanity.com/paper/2408.00818/) (2024)
- [Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas](https://mlsanity.com/paper/2408.06929/) (2024)
- [Cultural Evolution of Cooperation among LLM Agents](https://mlsanity.com/paper/2412.10270/) (2024)
- [Position: Theory of Mind Benchmarks are Broken for Large Language Models](https://mlsanity.com/paper/2412.19726/) (2024)
- [Network Formation and Dynamics Among Multi-LLMs](https://mlsanity.com/paper/2402.10659/) (2024)
- [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024)
- [Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents](https://mlsanity.com/paper/2502.13012/) (2025)
- [LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra](https://mlsanity.com/paper/2507.15815/) (2025)
- [Toward LLM-Agent-Based Modeling of Transportation Systems: A Conceptual Framework](https://mlsanity.com/paper/2412.06681/) (2024)
- [DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios](https://mlsanity.com/paper/2510.15501/) (2025)
- [Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue](https://mlsanity.com/paper/2409.08330/) (2024)
- [On Catastrophic Inheritance of Large Foundation Models](https://mlsanity.com/paper/2402.01909/) (2024)
- [Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation](https://mlsanity.com/paper/2404.09043/) (2024)
- [Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches](https://mlsanity.com/paper/2404.12744/) (2024)
- [From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs](https://mlsanity.com/paper/2502.17701/) (2025)
- [Are LLMs Socially Adaptive? Contrasting Belief Evolution in Large Language Models and Humans](https://mlsanity.com/paper/2410.10398/) (2024)
- [Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications](https://mlsanity.com/paper/2505.00049/) (2025)
- [Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research](https://mlsanity.com/paper/2506.01839/) (2025)
- [Latent Logic Tree Extraction for Event Sequence Explanation from LLMs](https://mlsanity.com/paper/2406.01124/) (2024)
- [Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values](https://mlsanity.com/paper/2502.00313/) (2025)
- [The Good, the Bad, and the Hulk-like GPT: Analyzing Emotional Decisions of Large Language Models in Cooperation and Bargaining Games](https://mlsanity.com/paper/2406.03299/) (2024)
- [Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies](https://mlsanity.com/paper/2504.19487/) (2025)
- [Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy](https://mlsanity.com/paper/2504.00241/) (2025)
- [To Mask or to Mirror: Human-AI Alignment in Collective Reasoning](https://mlsanity.com/paper/2510.01924/) (2025)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [Gender Dynamics and Homophily in a Social Network of LLM Agents](https://mlsanity.com/paper/2602.02606/) (2026)
- [Talk, Judge, Cooperate: Gossip-Driven Indirect Reciprocity in Self-Interested LLM Agents](https://mlsanity.com/paper/2602.07777/) (2026)
- [Simulating Influence Dynamics with LLM Agents](https://mlsanity.com/paper/2503.08709/) (2025)

## Abstract (arXiv)

As LLMs increasingly take on roles in human-AI interactions and autonomous AI systems, understanding their social behavior becomes important for informed use and continuous improvement. However, their behaviors in social interactions with humans and other agents, as well as the mechanisms shaping their responses, remain underexplored. To address this gap, we introduce a novel probabilistic framework, State-Understanding-Value-Action (SUVA), to systematically analyze LLM responses in social contexts based on their textual outputs (i.e., utterances). Using canonical behavioral economics games and social preference concepts relatable to LLM users, SUVA assesses LLMs' social behavior through both their final decisions and the response generation processes leading to those decisions. Our analysis of eight LLMs -- including two GPT, four LLaMA, and two Mistral models -- suggests that most models do not generate decisions aligned solely with self-interest; instead, they often produce responses that reflect social welfare considerations and display patterns consistent with direct and indirect reciprocity. Additionally, higher-capacity models more frequently display group identity effects. The SUVA framework also provides explainable tools -- including tree-based visualizations and probabilistic dependency analysis -- to elucidate how factors in LLMs' utterance-based reasoning influence their decisions. We demonstrate that utterance-based reasoning reliably predicts LLMs' final actions; references to altruism, fairness, and cooperation in the reasoning increase the likelihood of prosocial actions, while mentions of self-interest and competition reduce them. Overall, our framework enables practitioners to assess LLMs for applications involving social interactions, and provides researchers with a structured method to interpret how LLM behavior arises from utterance-based reasoning.