mlsanityData to 14 Oct 2026

A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios

arXiv 2412.03920 ↗First posted 5 Dec 2024cs.CL cs.AIcited by 23

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Game-theoretic scenarios have become pivotal in evaluating the social intelligence of Large Language Model (LLM)-based social agents. While numerous studies have explored these agents in such settings, there is a lack of a comprehensive survey summarizing the current progress. To address this gap, we systematically review existing research on LLM-based social agents within game-theoretic scenarios. Our survey organizes the findings into three core components: Game Framework, Social Agent, and Evaluation Protocol. The game framework encompasses diverse game scenarios, ranging from choice-focusing to communication-focusing games. The social agent part explores agents' preferences, beliefs, and reasoning abilities, as well as their interactions and synergistic effects on decision-making. The evaluation protocol covers both game-agnostic and game-specific metrics for assessing agent performance. Additionally, we analyze the performance of current social agents across various game scenarios. By reflecting on the current research and identifying future research directions, this survey provides insights to advance the development and evaluation of social agents in game-theoretic scenarios.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 103 of its 146 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

103 of 146 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information5 Aug
  2. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy9 Jul
  3. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents7 Jun
  4. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games26 Apr
  5. 2024A Survey on Large Language Model-Based Game Agents2 Apr
  6. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments18 Mar
  7. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization27 Feb
  8. 2024Enhance Reasoning for Large Language Models in the Game Werewolf4 Feb
  9. 2024SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models31 Jan
  10. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents19 Jan
  11. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach19 Dec
  12. 2023Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games1 Dec
4 more
  1. 2023MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration14 Nov
  2. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game29 Oct
  3. 2023AvalonBench: Evaluating LLMs Playing the Game of Avalon8 Oct
  4. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-429 Sep

Other references · 87

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  5. 2023Mistral 7Bcited by 3,164
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  8. 2024Qwen2 Technical Reportcited by 2,120
  9. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  10. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  11. 2023Cognitive Architectures for Language Agentscited by 429
  12. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
38 more
  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  2. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  3. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 233
  4. 2023Nash Learning from Human Feedbackcited by 224
  5. 2023Playing repeated games with Large Language Modelscited by 217
  6. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  7. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  8. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 207
  9. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  10. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  11. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  12. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 135
  13. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 120
  14. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  15. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
  16. 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysiscited by 100
  17. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  18. 2016Opponent Modeling in Deep Reinforcement Learningcited by 97
  19. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  20. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 92
  21. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 78
  22. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  23. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
  24. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 66
  25. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  26. 2023Strategic Reasoning with Language Modelscited by 64
  27. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 63
  28. 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 63
  29. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
  30. 2023Do LLM Agents Exhibit Social Behavior?cited by 53
  31. 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 51
  32. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  33. 2024Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Methodcited by 48
  34. 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 47
  35. 2024Werewolf Arena: A Case Study in LLM Evaluation via Social Deductioncited by 45
  36. 2024Emergence of Social Norms in Generative Agent Societies: Principles and Architecturecited by 41
  37. 2024Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Contextcited by 41
  38. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 37

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 9.

23 papers.

Sort:

By area:

  1. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  3. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 16
  4. 2025Reasoning Does Not Necessarily Improve Role-Playing Abilitycited by 14
  5. 2025Towards Online Multi-Modal Social Interaction Understandingcited by 12
  6. 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 12
  7. 2026Evaluating Collective Behaviour of Hundreds of LLM Agentscited by 8
  8. 2025From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactionscited by 7
  9. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Promptscited by 5
  10. 2026Omni-MMSI: Toward Identity-attributed Social Interaction Understandingcited by 4
  11. 2026SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attributioncited by 4
  12. 2025ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiationcited by 3
11 more
  1. 2025AI Agents Should be Regulated Based on the Extent of Their Autonomous Operationscited by 2
  2. 2025Aligning VLM Assistants with Personalized Situated Cognitioncited by 2
  3. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  4. 2025InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Stylescited by 2
  5. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
  6. 2026Socially-Weighted Alignment: A Game-Theoretic Framework for Multi-Agent LLM Systemscited by 2
  7. 2025CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMscited by 1
  8. 2026Assessing mentalization in humans and large language modelscited by 1
  9. 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Playcited by 0
  10. 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Gamecited by 0
  11. 2026Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Game-Theoretic Lens on LLM-based Multi-Agent Systems
  2. 2026Communicate-Predict-Act: Evaluating Social Intelligence of Agents
  3. 2024AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
  4. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
  5. 2025Grounding Natural Language for Multi-agent Decision-Making with Multi-agentic LLMs
  6. 2024Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games
  7. 2025Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play
  8. 2025Playing games with Large language models: Randomness and strategy