# Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic

arXiv: https://arxiv.org/abs/1401.5577 · HTML: https://mlsanity.com/paper/1401.5577/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2014.

## Built on by (most cited first, 13 of 13)

- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Evaluating LLMs in Open-Source Games](https://mlsanity.com/paper/2512.00371/) (2025)
- [Similarity-based cooperative equilibrium](https://mlsanity.com/paper/2211.14468/) (2022)
- [Recursive Joint Simulation in Games](https://mlsanity.com/paper/2402.08128/) (2024)
- [A dataset of questions on decision-theoretic reasoning in Newcomb-like problems](https://mlsanity.com/paper/2411.10588/) (2024)
- [Game-Theoretic Models of Moral and Other-Regarding Agents (extended abstract)](https://mlsanity.com/paper/2106.11503/) (2021)
- [A Theory of Bounded Inductive Rationality](https://mlsanity.com/paper/2307.05068/) (2023)
- [A game theory for foundation models shows new paths to rational cooperation through similarity inference](https://mlsanity.com/paper/2608.03958/) (2026)
- [Policy-Conditioned Policies for Multi-Agent Task Solving](https://mlsanity.com/paper/2512.21024/) (2025)
- [Parametric Open Source Games](https://mlsanity.com/paper/2606.27068/) (2026)
- [The Open-Strategy Dictator Game: Cooperation Under Mutual Transparency](https://mlsanity.com/paper/2608.14913/) (2026)
- [Math for AI safety: an invitation for mathematicians](https://mlsanity.com/paper/2609.15289/) (2026)
- [On the Computational Tractability of Robust Bandits](https://mlsanity.com/paper/2610.08740/) (2026)

## Abstract (arXiv)

We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties.