mlsanityData to 14 Oct 2026

Overcoming the Machine Penalty with Imperfectly Fair AI Agents

arXiv 2410.03724 ↗First posted 29 Sep 2024cs.HC cs.AI cs.GTcited by 2

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Despite rapid technological progress, effective human-machine cooperation remains a significant challenge. Humans tend to cooperate less with machines than with fellow humans, a phenomenon known as the machine penalty. Here, we show that artificial intelligence (AI) agents powered by large language models can overcome this penalty in social dilemma games with communication. In a pre-registered experiment with 1,152 participants, we deploy AI agents exhibiting three distinct personas: selfish, cooperative, and fair. However, only fair agents elicit human cooperation at rates comparable to human-human interactions. Analysis reveals that fair agents, similar to human participants, occasionally break pre-game cooperation promises, but nonetheless effectively establish cooperation as a social norm. These results challenge the conventional wisdom of machines as altruistic assistants or rational actors. Instead, our study highlights the importance of AI agents reflecting the nuanced complexity of human social behaviors -- imperfect yet driven by deeper social cognitive processes.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 6 of its 84 references that are in the map. First posted on arXiv 29 Sep 2024.

Builds on

6 of 84 references.

References · 6

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  5. 2023A Comprehensive Overview of Large Language Modelscited by 648
  6. 2023Playing repeated games with Large Language Modelscited by 217

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas
  2. 2024Cultural Evolution of Cooperation among LLM Agents
  3. 2025When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemma
  4. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agents
  5. 2024Can Machines Think Like Humans? A Behavioral Evaluation of LLM Agents in Dictator Games
  6. 2026Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
  7. 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Game
  8. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents