# Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing

arXiv: https://arxiv.org/abs/2309.05898 · HTML: https://mlsanity.com/paper/2309.05898/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 50 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking](https://mlsanity.com/paper/2305.15066/) (2023)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Thinking Fast and Slow in Large Language Models](https://mlsanity.com/paper/2212.05206/) (2022)
- [GPT-4 Can't Reason](https://mlsanity.com/paper/2308.03762/) (2023)
- [Mind meets machine: Unravelling GPT-4's cognitive psychology](https://mlsanity.com/paper/2303.11436/) (2023)

## Built on by (most cited first, 30 of 51)

- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Do LLM Agents Have Regret? A Case Study in Online Learning and Games](https://mlsanity.com/paper/2403.16843/) (2024)
- [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [LLM-Based Social Simulations Require a Boundary](https://mlsanity.com/paper/2506.19806/) (2025)
- [AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactions](https://mlsanity.com/paper/2602.06008/) (2026)
- [Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulations](https://mlsanity.com/paper/2410.13204/) (2024)
- [Be.FM: Open Foundation Models for Human Behavior](https://mlsanity.com/paper/2505.23058/) (2025)
- [Beyond Nash Equilibrium: Bounded Rationality of LLMs and humans in Strategic Decision-making](https://mlsanity.com/paper/2506.09390/) (2025)
- [The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind](https://mlsanity.com/paper/2509.20393/) (2025)
- [Evaluating Collective Behaviour of Hundreds of LLM Agents](https://mlsanity.com/paper/2602.16662/) (2026)
- [Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits](https://mlsanity.com/paper/2605.18890/) (2026)
- [Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models](https://mlsanity.com/paper/2408.05241/) (2024)
- [Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy](https://mlsanity.com/paper/2508.07485/) (2025)
- [Comparing the Framing Effect in Humans and LLMs on Naturally Occurring Texts](https://mlsanity.com/paper/2502.17091/) (2025)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Harnessing Language for Coordination: A Framework and Benchmark for LLM-Driven Multi-Agent Control](https://mlsanity.com/paper/2412.11761/) (2024)
- [Framing the Game: How Context Shapes LLM Decision-Making](https://mlsanity.com/paper/2503.04840/) (2025)
- [Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?](https://mlsanity.com/paper/2508.05670/) (2025)
- [AI Agents as Policymakers in Simulated Epidemics](https://mlsanity.com/paper/2601.04245/) (2026)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions](https://mlsanity.com/paper/2605.00226/) (2026)
- [Synthetic Social Media Influence Experimentation via an Agentic Reinforcement Learning Large Language Model Bot](https://mlsanity.com/paper/2411.19635/) (2024)
- [Social preferences with unstable interactive reasoning: Large language models in economic trust games](https://mlsanity.com/paper/2505.17053/) (2025)
- [Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Markets](https://mlsanity.com/paper/2505.20120/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators](https://mlsanity.com/paper/2605.16575/) (2026)
- [CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs](https://mlsanity.com/paper/2508.11944/) (2025)
- [Psychologically Enhanced AI Agents](https://mlsanity.com/paper/2509.04343/) (2025)
- [Payoff scaling shapes cooperation in LLM agents across languages](https://mlsanity.com/paper/2601.19082/) (2026)

## Abstract (arXiv)

This paper investigates the strategic decision-making capabilities of three Large Language Models (LLMs): GPT-3.5, GPT-4, and LLaMa-2, within the framework of game theory. Utilizing four canonical two-player games -- Prisoner's Dilemma, Stag Hunt, Snowdrift, and Prisoner's Delight -- we explore how these models navigate social dilemmas, situations where players can either cooperate for a collective benefit or defect for individual gain. Crucially, we extend our analysis to examine the role of contextual framing, such as diplomatic relations or casual friendships, in shaping the models' decisions. Our findings reveal a complex landscape: while GPT-3.5 is highly sensitive to contextual framing, it shows limited ability to engage in abstract strategic reasoning. Both GPT-4 and LLaMa-2 adjust their strategies based on game structure and context, but LLaMa-2 exhibits a more nuanced understanding of the games' underlying mechanics. These results highlight the current limitations and varied proficiencies of LLMs in strategic decision-making, cautioning against their unqualified use in tasks requiring complex strategic reasoning.