mlsanityData to 7 Oct 2026

Measuring an artificial intelligence agent's trust in humans using machine incentives

arXiv 2212.13371 ↗First posted 27 Dec 2022cs.AI cs.HC econ.GNcited by 5

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Scientists and philosophers have debated whether humans can trust advanced artificial intelligence (AI) agents to respect humanity's best interests. Yet what about the reverse? Will advanced AI agents trust humans? Gauging an AI agent's trust in humans is challenging because--absent costs for dishonesty--such agents might respond falsely about their trust in humans. Here we present a method for incentivizing machine decisions without altering an AI agent's underlying algorithms or goal orientation. In two separate experiments, we then employ this method in hundreds of trust games between an AI agent (a Large Language Model (LLM) from OpenAI) and a human experimenter (author TJ). In our first experiment, we find that the AI agent decides to trust humans at higher rates when facing actual incentives than when making hypothetical decisions. Our second experiment replicates and extends these findings by automating game play and by homogenizing question wording. We again observe higher rates of trust when the AI agent faces real incentives. Across both experiments, the AI agent's trust decisions appear unrelated to the magnitude of stakes. Furthermore, to address the possibility that the AI agent's trust decisions reflect a preference for uncertainty, the experiments include two conditions that present the AI agent with a non-social decision task that provides the opportunity to choose a certain or uncertain option; in those conditions, the AI agent consistently chooses the certain option. Our experiments suggest that one of the most advanced AI language models to date alters its social behavior in response to incentives and displays behavior consistent with trust toward a human interlocutor when incentivized.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 5 of its 29 references that are in the map. First posted on arXiv 27 Dec 2022.

Builds on

5 of 29 references.

References · 5

  1. 2020Artificial Intelligence, Values and Alignmentcited by 443
  2. 2022Using cognitive psychology to understand GPT-3cited by 389
  3. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  4. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
  5. 2021Problematic Machine Behavior: A Systematic Literature Review of Algorithm Auditscited by 20

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 0; 2025: 1.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents
  2. 2026Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems
  3. 2025Human aversion? Do AI Agents Judge Identity More Harshly Than Performance
  4. 2024Can Large Language Model Agents Simulate Human Trust Behavior?
  5. 2026From Role to Person: Trust Calibration Challenges in Twin Agents
  6. 2026Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts
  7. 2023Evidence of behavior consistent with self-interest and altruism in an artificially intelligent agent
  8. 2025When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemma