mlsanityData to 7 Oct 2026

Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing

arXiv 2309.05898 ↗First posted 12 Sep 2023cs.GT cs.AI cs.CYcited by 49

Not on the map. Why?

Cited by 22 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper investigates the strategic decision-making capabilities of three Large Language Models (LLMs): GPT-3.5, GPT-4, and LLaMa-2, within the framework of game theory. Utilizing four canonical two-player games -- Prisoner's Dilemma, Stag Hunt, Snowdrift, and Prisoner's Delight -- we explore how these models navigate social dilemmas, situations where players can either cooperate for a collective benefit or defect for individual gain. Crucially, we extend our analysis to examine the role of contextual framing, such as diplomatic relations or casual friendships, in shaping the models' decisions. Our findings reveal a complex landscape: while GPT-3.5 is highly sensitive to contextual framing, it shows limited ability to engage in abstract strategic reasoning. Both GPT-4 and LLaMa-2 adjust their strategies based on game structure and context, but LLaMa-2 exhibits a more nuanced understanding of the games' underlying mechanics. These results highlight the current limitations and varied proficiencies of LLMs in strategic decision-making, cautioning against their unqualified use in tasks requiring complex strategic reasoning.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 10 of its 50 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

10 of 50 references.

References · 10

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  5. 2023Playing repeated games with Large Language Modelscited by 213
  6. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  7. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  8. 2022Thinking Fast and Slow in Large Language Modelscited by 136
  9. 2023GPT-4 Can't Reasoncited by 39
  10. 2023Mind meets machine: Unravelling GPT-4's cognitive psychologycited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 18; 2026: 25.

49 papers.

Sort:

By area:

  1. 2025Agentic Large Language Models, a surveycited by 105
  2. 2024Do LLM Agents Have Regret? A Case Study in Online Learning and Gamescited by 33
  3. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  4. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 14
  5. 2024Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulationscited by 11
  6. 2025LLM-Based Social Simulations Require a Boundarycited by 11
  7. 2026AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactionscited by 11
  8. 2025Beyond Nash Equilibrium: Bounded Rationality of LLMs and humans in Strategic Decision-makingcited by 8
  9. 2024Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Modelscited by 7
  10. 2025The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blindcited by 7
  11. 2025Comparing the Framing Effect in Humans and LLMs on Naturally Occurring Textscited by 6
  12. 2025Be.FM: Open Foundation Models for Human Behaviorcited by 6
37 more
  1. 2025Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacycited by 6
  2. 2026Evaluating Collective Behaviour of Hundreds of LLM Agentscited by 6
  3. 2026Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Auditscited by 6
  4. 2025AI Agent Behavioral Sciencecited by 5
  5. 2024Harnessing Language for Coordination: A Framework and Benchmark for LLM-Driven Multi-Agent Controlcited by 4
  6. 2025Framing the Game: How Context Shapes LLM Decision-Makingcited by 4
  7. 2026AI Agents as Policymakers in Simulated Epidemicscited by 3
  8. 2026Contextual Drag: How Errors in the Context Affect LLM Reasoningcited by 3
  9. 2024Synthetic Social Media Influence Experimentation via an Agentic Reinforcement Learning Large Language Model Botcited by 2
  10. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
  11. 2025Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?cited by 2
  12. 2026Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actionscited by 2
  13. 2025Social preferences with unstable interactive reasoning: Large language models in economic trust gamescited by 1
  14. 2025CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMscited by 1
  15. 2025Psychologically Enhanced AI Agentscited by 1
  16. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  17. 2026Payoff scaling shapes cooperation in LLM agents across languagescited by 1
  18. 2026Framing Effects in Independent-Agent Large Language Models: A Cross-Family Behavioral Analysiscited by 1
  19. 2026Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agentscited by 1
  20. 2026Cooperative Profiles Predict Multi-Agent LLM Team Performance in AI for Science Workflowscited by 1
  21. 2026Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignmentcited by 1
  22. 2026Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agentscited by 1
  23. 2026Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectivescited by 1
  24. 2026Emergence of Reputation-Based Cooperation in LLM Agentscited by 1
  25. 2025CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimizationcited by 0
  26. 2025ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspectivecited by 0
  27. 2026When Identity Overrides Incentives: Representational Choices as Governance Decisions in Multi-Agent LLM Systemscited by 0
  28. 2026On Emotion-Sensitive Decision Making of Small Language Model Agentscited by 0
  29. 2026Counterparty Modeling is Not Strategy: The Limits of LLM Negotiatorscited by 0
  30. 2026GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Modelscited by 0
  31. 2026Measuring Behavior Portability in Large Language Modelscited by 0
  32. 2026BehaviorBench: Benchmarking Foundation Models for Behavioral Science Taskscited by 0
  33. 2026The Politician, the Liar, and the Obedient Worker: Emerging Behavior of LLM Agents in Hierarchical Gamescited by 0
  34. 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
  35. 2026Benchmarking large language model agent societies against human behavioural distributionscited by 0
  36. 2026Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogatescited by 0
  37. 2026Cheap Talk Stabilizes Strategic Interaction in LLM Agentscited by 0