# When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games

arXiv: https://arxiv.org/abs/2007.11338 · HTML: https://mlsanity.com/paper/2007.11338/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory](https://mlsanity.com/paper/2504.14325/) (2025)
- [Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas](https://mlsanity.com/paper/2402.17270/) (2024)
- [Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents](https://mlsanity.com/paper/2504.08640/) (2025)
- [Understanding LLM Agent Behaviours via Game Theory: Strategy Recognition, Biases and Multi-Agent Dynamics](https://mlsanity.com/paper/2512.07462/) (2025)
- [Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation](https://mlsanity.com/paper/2403.09510/) (2024)
- [Strategic Communication and Language Bias in Multi-Agent LLM Coordination](https://mlsanity.com/paper/2508.00032/) (2025)
- [Social physics in the age of artificial intelligence](https://mlsanity.com/paper/2603.16900/) (2026)
- [Evolutionary mechanisms that promote cooperation may not promote social welfare](https://mlsanity.com/paper/2408.05373/) (2024)
- [Understanding Emergent Behaviours in Multi-Agent Systems with Evolutionary Game Theory](https://mlsanity.com/paper/2205.07369/) (2022)
- [Payoff scaling shapes cooperation in LLM agents across languages](https://mlsanity.com/paper/2601.19082/) (2026)
- [Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems](https://mlsanity.com/paper/2603.24742/) (2026)
- [The evolutionary advantage of guilt: co-evolution of social and non-social guilt in structured populations](https://mlsanity.com/paper/2302.09859/) (2023)
- [The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance](https://mlsanity.com/paper/2606.28710/) (2026)

## Abstract (arXiv)

The actions of intelligent agents, such as chatbots, recommender systems, and virtual assistants are typically not fully transparent to the user. Consequently, using such an agent involves the user exposing themselves to the risk that the agent may act in a way opposed to the user's goals. It is often argued that people use trust as a cognitive shortcut to reduce the complexity of such interactions. Here we formalise this by using the methods of evolutionary game theory to study the viability of trust-based strategies in repeated games. These are reciprocal strategies that cooperate as long as the other player is observed to be cooperating. Unlike classic reciprocal strategies, once mutual cooperation has been observed for a threshold number of rounds they stop checking their co-player's behaviour every round, and instead only check with some probability. By doing so, they reduce the opportunity cost of verifying whether the action of their co-player was actually cooperative. We demonstrate that these trust-based strategies can outcompete strategies that are always conditional, such as Tit-for-Tat, when the opportunity cost is non-negligible. We argue that this cost is likely to be greater when the interaction is between people and intelligent agents, because of the reduced transparency of the agent. Consequently, we expect people to use trust-based strategies more frequently in interactions with intelligent agents. Our results provide new, important insights into the design of mechanisms for facilitating interactions between humans and intelligent agents, where trust is an essential factor.