# Cooperating with Machines

arXiv: https://arxiv.org/abs/1703.06207 · HTML: https://mlsanity.com/paper/1703.06207/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2017.

## Built on by (most cited first, 30 of 39)

- [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://mlsanity.com/paper/2204.13988/) (2022)
- [Welfare Diplomacy: Benchmarking Language Model Cooperation](https://mlsanity.com/paper/2310.08901/) (2023)
- [Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AI](https://mlsanity.com/paper/2105.05424/) (2021)
- [Continuous Coordination As a Realistic Scenario for Lifelong Learning](https://mlsanity.com/paper/2103.03216/) (2021)
- [Performative Reinforcement Learning](https://mlsanity.com/paper/2207.00046/) (2022)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas](https://mlsanity.com/paper/2402.17270/) (2024)
- [AI-enhanced Collective Intelligence](https://mlsanity.com/paper/2403.10433/) (2024)
- [Spontaneous Giving and Calculated Greed in Language Models](https://mlsanity.com/paper/2502.17720/) (2025)
- [Conceptual Framework for Autonomous Cognitive Entities](https://mlsanity.com/paper/2310.06775/) (2023)
- [Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study](https://mlsanity.com/paper/2112.07774/) (2021)
- [Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gain](https://mlsanity.com/paper/2401.16444/) (2024)
- [Social physics in the age of artificial intelligence](https://mlsanity.com/paper/2603.16900/) (2026)
- [Over-communicate no more: Situated RL agents learn concise communication protocols](https://mlsanity.com/paper/2211.01480/) (2022)
- [The Science Fiction Science Method](https://mlsanity.com/paper/2508.03430/) (2025)
- [Modeling Human-AI Team Decision Making](https://mlsanity.com/paper/2201.02759/) (2022)
- [Pavlovian Signalling with General Value Functions in Agent-Agent Temporal Decision Making](https://mlsanity.com/paper/2201.03709/) (2022)
- [The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents](https://mlsanity.com/paper/2203.09498/) (2022)
- [Incorporating Human Flexibility through Reward Preferences in Human-AI Teaming](https://mlsanity.com/paper/2312.14292/) (2023)
- [Rethinking the Illusion of Thinking](https://mlsanity.com/paper/2507.01231/) (2025)
- [Cooperation Through Indirect Reciprocity in Child-Robot Interactions](https://mlsanity.com/paper/2512.20621/) (2025)
- [Balancing Adaptability and Non-exploitability in Repeated Games](https://mlsanity.com/paper/2112.10314/) (2021)
- [Effect of Adaptation Rate and Cost Display in a Human-AI Interaction Game](https://mlsanity.com/paper/2408.14640/) (2024)
- [An AI Theory of Mind Will Enhance Our Collective Intelligence](https://mlsanity.com/paper/2411.09168/) (2024)
- [Human-AI Collaboration: Trade-offs Between Performance and Preferences](https://mlsanity.com/paper/2503.00248/) (2025)
- [The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis](https://mlsanity.com/paper/2509.10297/) (2025)
- [Towards Controllable Agent in MOBA Games with Generative Modeling](https://mlsanity.com/paper/2112.08093/) (2021)
- [Cooperative Artificial Intelligence](https://mlsanity.com/paper/2202.09859/) (2022)
- [Stress Propagation in Human-Robot Teams Based on Computational Logic Model](https://mlsanity.com/paper/2211.04056/) (2022)
- [Continually Learned Pavlovian Signalling Without Forgetting for Human-in-the-Loop Robotic Control](https://mlsanity.com/paper/2305.14365/) (2023)

## Abstract (arXiv)

Since Alan Turing envisioned Artificial Intelligence (AI) [1], a major driving force behind technical progress has been competition with human cognition. Historical milestones have been frequently associated with computers matching or outperforming humans in difficult cognitive tasks (e.g. face recognition [2], personality classification [3], driving cars [4], or playing video games [5]), or defeating humans in strategic zero-sum encounters (e.g. Chess [6], Checkers [7], Jeopardy! [8], Poker [9], or Go [10]). In contrast, less attention has been given to developing autonomous machines that establish mutually cooperative relationships with people who may not share the machine's preferences. A main challenge has been that human cooperation does not require sheer computational power, but rather relies on intuition [11], cultural norms [12], emotions and signals [13, 14, 15, 16], and pre-evolved dispositions toward cooperation [17], common-sense mechanisms that are difficult to encode in machines for arbitrary contexts. Here, we combine a state-of-the-art machine-learning algorithm with novel mechanisms for generating and acting on signals to produce a new learning algorithm that cooperates with people and other machines at levels that rival human cooperation in a variety of two-player repeated stochastic games. This is the first general-purpose algorithm that is capable, given a description of a previously unseen game environment, of learning to cooperate with people within short timescales in scenarios previously unanticipated by algorithm designers. This is achieved without complex opponent modeling or higher-order theories of mind, thus showing that flexible, fast, and general human-machine cooperation is computationally achievable using a non-trivial, but ultimately simple, set of algorithmic mechanisms.