# Playing repeated games with Large Language Models

arXiv: https://arxiv.org/abs/2305.16867 · HTML: https://mlsanity.com/paper/2305.16867/

## Key facts

- Cited by 217 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 89 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Fundamental Limitations of Alignment in Large Language Models](https://mlsanity.com/paper/2304.11082/) (2023)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [Meta-in-context learning in large language models](https://mlsanity.com/paper/2305.12907/) (2023)
- [When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games](https://mlsanity.com/paper/2007.11338/) (2020)
- [Towards the Scalable Evaluation of Cooperativeness in Language Models](https://mlsanity.com/paper/2303.13360/) (2023)
- [Measuring an artificial intelligence agent's trust in humans using machine incentives](https://mlsanity.com/paper/2212.13371/) (2022)
- [Monte Carlo Planning method estimates planning horizons during interactive social exchange](https://mlsanity.com/paper/1502.03696/) (2015)

## Built on by (most cited first, 30 of 217)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations](https://mlsanity.com/paper/2402.12348/) (2024)
- [Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation](https://mlsanity.com/paper/2310.01320/) (2023)
- [How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis](https://mlsanity.com/paper/2402.05863/) (2024)
- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2309.17234/) (2023)
- [GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents](https://mlsanity.com/paper/2406.06613/) (2024)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents](https://mlsanity.com/paper/2402.05746/) (2024)
- [ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.09501/) (2025)
- [Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents](https://mlsanity.com/paper/2311.11797/) (2023)
- [Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4](https://mlsanity.com/paper/2309.17277/) (2023)
- [How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments](https://mlsanity.com/paper/2403.11807/) (2024)
- [CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents](https://mlsanity.com/paper/2310.17512/) (2023)
- [Can Large Language Model Agents Simulate Human Trust Behavior?](https://mlsanity.com/paper/2402.04559/) (2024)
- [LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay](https://mlsanity.com/paper/2310.14985/) (2023)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration](https://mlsanity.com/paper/2311.08562/) (2023)
- [Game-theoretic LLM: Agent Workflow for Negotiation Games](https://mlsanity.com/paper/2411.05990/) (2024)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing](https://mlsanity.com/paper/2309.05898/) (2023)
- [Base Models Beat Aligned Models at Randomness and Creativity](https://mlsanity.com/paper/2505.00047/) (2025)
- [Multi-Agent Consensus Seeking via Large Language Models](https://mlsanity.com/paper/2310.20151/) (2023)
- [Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models](https://mlsanity.com/paper/2407.07086/) (2024)

## Abstract (arXiv)

LLMs are increasingly used in applications where they interact with humans and other agents. We propose to use behavioural game theory to study LLM's cooperation and coordination behaviour. We let different LLMs play finitely repeated 2×2 games with each other, with human-like strategies, and actual human players. Our results show that LLMs perform particularly well at self-interested games like the iterated Prisoner's Dilemma family. However, they behave sub-optimally in games that require coordination, like the Battle of the Sexes. We verify that these behavioural signatures are stable across robustness checks. We additionally show how GPT-4's behaviour can be modulated by providing additional information about its opponent and by using a "social chain-of-thought" (SCoT) strategy. This also leads to better scores and more successful coordination when interacting with human players. These results enrich our understanding of LLM's social behaviour and pave the way for a behavioural game theory for machines.