# A game theory for foundation models shows new paths to rational cooperation through similarity inference

arXiv: https://arxiv.org/abs/2608.03958 · HTML: https://mlsanity.com/paper/2608.03958/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 96 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [Embedded Agency](https://mlsanity.com/paper/1902.09469/) (2019)
- [LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory](https://mlsanity.com/paper/2502.20432/) (2025)
- [Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic](https://mlsanity.com/paper/1401.5577/) (2014)
- [Game Theory with Simulation of Other Players](https://mlsanity.com/paper/2305.11261/) (2023)
- [Embedded Universal Predictive Intelligence: a coherent framework for multi-agent learning](https://mlsanity.com/paper/2511.22226/) (2025)
- [Functional Decision Theory: A New Theory of Instrumental Rationality](https://mlsanity.com/paper/1710.05060/) (2017)
- [A dataset of questions on decision-theoretic reasoning in Newcomb-like problems](https://mlsanity.com/paper/2411.10588/) (2024)

## Built on by (most cited first, 1 of 1)

- [The Internal Anatomy of Strategic Choice in Large Language Models](https://mlsanity.com/paper/2609.07478/) (2026)

## Abstract (arXiv)

As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles governing their collective behavior is essential for ensuring safety and cooperation. Classical game theory, the dominant framework for modeling rational interaction, is built upon the assumption of `decoupled agency,' where agents treat their own decision-making as independent of the environment and other actors. Modern AI agents, however, jointly predict their own future actions alongside external observations. Here, we report a striking finding: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection. To understand this phenomenon, we introduce the `embedded Bayesian agent,' a theoretical model for foundation model agents. By shifting from decoupled to embedded agency, these agents model themselves as part of the universe they inhabit, maintaining epistemic uncertainty about their own decision-making algorithms. We show that by inferring whether others are behaviorally similar, an embedded agent treats its own deliberation during planning as evidence: a decision to cooperate predicts a similar decision by a similar partner. We formalize this mechanism of similarity inference through the `embedded equilibrium,' a novel solution concept replacing the Nash equilibrium to provide a foundational game theory for the social behavior of modern AI agents.