# Evaluating LLMs in Open-Source Games

arXiv: https://arxiv.org/abs/2512.00371 · HTML: https://mlsanity.com/paper/2512.00371/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Builds on (most cited first, 23 of 23)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [MLGym: A New Framework and Benchmark for Advancing AI Research Agents](https://mlsanity.com/paper/2502.14499/) (2025)
- [Strategic Reasoning with Language Models](https://mlsanity.com/paper/2305.19165/) (2023)
- [Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?](https://mlsanity.com/paper/2406.13605/) (2024)
- [Maintaining cooperation in complex social dilemmas using deep reinforcement learning](https://mlsanity.com/paper/1707.01068/) (2017)
- [Managing extreme AI risks amid rapid progress](https://mlsanity.com/paper/2310.17688/) (2023)
- [Finding Friend and Foe in Multi-Agent Games](https://mlsanity.com/paper/1906.02330/) (2019)
- [Multi-agent Reinforcement Learning: A Comprehensive Survey](https://mlsanity.com/paper/2312.10256/) (2023)
- [Theory of Minds: Understanding Behavior in Groups Through Inverse Planning](https://mlsanity.com/paper/1901.06085/) (2019)
- [Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma](https://mlsanity.com/paper/2501.16173/) (2025)
- [Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination](https://mlsanity.com/paper/2504.12714/) (2025)
- [Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic](https://mlsanity.com/paper/1401.5577/) (2014)
- [Game Theory with Simulation of Other Players](https://mlsanity.com/paper/2305.11261/) (2023)
- [Cooperative and uncooperative institution designs: Surprises and problems in open-source game theory](https://mlsanity.com/paper/2208.07006/) (2022)
- [Modeling Others' Minds as Code](https://mlsanity.com/paper/2510.01272/) (2025)
- [Game Theory with Translucent Players](https://mlsanity.com/paper/1310.6438/) (2013)
- [From Code to Play: Benchmarking Program Search for Games Using Large Language Models](https://mlsanity.com/paper/2412.04057/) (2024)
- [Similarity-based cooperative equilibrium](https://mlsanity.com/paper/2211.14468/) (2022)
- [Learning in two-player games between transparent opponents](https://mlsanity.com/paper/2012.02671/) (2020)

## Built on by (most cited first, 5 of 5)

- [GUI Agents for Continual Game Generation](https://mlsanity.com/paper/2605.28258/) (2026)
- [Policy-Conditioned Policies for Multi-Agent Task Solving](https://mlsanity.com/paper/2512.21024/) (2025)
- [RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty](https://mlsanity.com/paper/2602.12424/) (2026)
- [The Open-Strategy Dictator Game: Cooperation Under Mutual Transparency](https://mlsanity.com/paper/2608.14913/) (2026)
- [Tapes Together Strong: The Co-evolution of Computation and Cooperation](https://mlsanity.com/paper/2609.10817/) (2026)

## Abstract (arXiv)

Large Language Models' (LLMs) programming capabilities enable their participation in open-source games: a game-theoretic setting in which players submit computer programs in lieu of actions. These programs offer numerous advantages, including interpretability, inter-agent transparency, and formal verifiability; additionally, they enable program equilibria, solutions that leverage the transparency of code and are inaccessible within normal-form settings. We evaluate the capabilities of leading open- and closed-weight LLMs to predict and classify program strategies and evaluate features of the approximate program equilibria reached by LLM agents in dyadic and evolutionary settings. We identify the emergence of payoff-maximizing, cooperative, and deceptive strategies, characterize the adaptation of mechanisms within these programs over repeated open-source games, and analyze their comparative evolutionary fitness. We find that open-source games serve as a viable environment to study and steer the emergence of cooperative strategy in multi-agent dilemmas.