mlsanityData to 7 Oct 2026

Payoff scaling shapes cooperation in LLM agents across languages

arXiv 2601.19082 ↗First posted 27 Jan 2026cs.AI cs.CL cs.GTcited by 1

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users. Whether they cooperate in such settings is no longer just an academic question, but a central issue for AI governance. We approach it from a strategic-behaviour angle, asking how two everyday levers - the size of what is at stake, and the language in which the interaction is described - shape the strategies LLMs adopt in a repeated Prisoner's Dilemma. Rather than reading cooperation off raw action counts, we train supervised classifiers to recognise the canonical strategies of repeated games (always cooperate, always defect, Tit-for-Tat, Win-Stay-Lose-Shift) and use them as a lens onto LLM behaviour. To know what the strategy distribution should look like under the same payoffs, we derive an evolutionary game theory (EGT) baseline and compare it with the LLM data. The two outcomes disagree in a revealing way: as stakes grow, evolutionary theory predicts that defection should take over the population, yet LLMs move in the opposite direction, becoming more cooperative - a signature, we argue, of alignment training and the human-like reasoning patterns LLMs inherit from their training data. We further show that this picture is not particular to frontier-scale, proprietary models: it also occurs with three open-weight smaller LLMs. Overall, our analysis highlights that payoff design and linguistic framing are powerful but under-explored levers for steering LLM behaviour, with direct implications for evaluating, aligning, and governing multi-agent AI systems deployed in high-stakes, multilingual environments.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 15 of its 55 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

15 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents11 Apr

Other references · 14

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2023Playing repeated games with Large Language Modelscited by 213
  5. 2025Multi-Agent Risks from Advanced AIcited by 150
  6. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 91
  7. 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 49
  8. 2020When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated gamescited by 14
  9. 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 14
  10. 2025Media and responsible AI governance: a game-theoretic and LLM analysiscited by 11
  11. 2022Inferring Strategies from Observations in Long Iterated Prisoner's Dilemma Experimentscited by 6
  12. 2026Mapping Human Anti-collusion Mechanisms to Multi-agent AI Systemscited by 4
2 more
  1. 2025How large language models judge and influence human cooperationcited by 3
  2. 2018Functional Dynamics by Intention Recognition in Iterated Gamescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Understanding LLM Agent Behaviours via Game Theory: Strategy Recognition, Biases and Multi-Agent Dynamics
  2. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
  3. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies
  4. 2026Collective cooperation without individual fidelity in LLM agents
  5. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
  6. 2026Evaluating Collective Behaviour of Hundreds of LLM Agents
  7. 2025Collaboration and Conflict between Humans and Language Models through the Lens of Game Theory
  8. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration