# Cooperation and Control in Delegation Games

arXiv: https://arxiv.org/abs/2402.15821 · HTML: https://mlsanity.com/paper/2402.15821/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Builds on 10 of its 59 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Open Problems in Cooperative AI](https://mlsanity.com/paper/2012.08630/) (2020)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Quantifying Differences in Reward Functions](https://mlsanity.com/paper/2006.13900/) (2020)
- [Multi-Agent Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1907.13220/) (2019)
- [Occam's razor is insufficient to infer the preferences of irrational agents](https://mlsanity.com/paper/1712.05812/) (2017)
- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [Computational Rationalization: The Inverse Equilibrium Problem](https://mlsanity.com/paper/1308.3506/) (2013)
- [STARC: A General Framework For Quantifying Differences Between Reward Functions](https://mlsanity.com/paper/2309.15257/) (2023)

## Built on by (most cited first, 2 of 2)

- [Collaborating with GenAI: Incentives and Replacements](https://mlsanity.com/paper/2508.20213/) (2025)
- [Causal Foundations of Collective Agency](https://mlsanity.com/paper/2605.00248/) (2026)

## Abstract (arXiv)

Many settings of interest involving humans and machines -- from virtual personal assistants to autonomous vehicles -- can naturally be modelled as principals (humans) delegating to agents (machines), which then interact with each other on their principals' behalf. We refer to these multi-principal, multi-agent scenarios as delegation games. In such games, there are two important failure modes: problems of control (where an agent fails to act in line their principal's preferences) and problems of cooperation (where the agents fail to work well together). In this paper we formalise and analyse these problems, further breaking them down into issues of alignment (do the players have similar preferences?) and capabilities (how competent are the players at satisfying those preferences?). We show -- theoretically and empirically -- how these measures determine the principals' welfare, how they can be estimated using limited observations, and thus how they might be used to help us design more aligned and cooperative AI systems.