mlsanityData to 14 Oct 2026

Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play

arXiv 2508.03368 ↗First posted 5 Aug 2025cs.AI cs.GTcited by 11

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

The Game Reasoning Arena library provides a framework for evaluating the decision making abilities of large language models (LLMs) through strategic board games implemented in Google OpenSpiel library. The framework enables systematic comparisons between LLM based agents and other agents (random, heuristic, reinforcement learning agents, etc.) in various game scenarios by wrapping multiple board and matrix games and supporting different agent types. It integrates API access to models via liteLLM, local model deployment via vLLM, and offers distributed execution through Ray. This paper summarises the library structure, key characteristics, and motivation of the repository, highlighting how it contributes to the empirical evaluation of the reasoning of LLM and game theoretic behaviour.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 4 of its 6 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

4 of 6 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024GameArena: Evaluating LLM Reasoning through Live Computer Games9 Dec
  2. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents7 Jun

Other references · 2

  1. 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 55
  2. 2024Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboardcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2025Code World Models for General Game Playingcited by 26
  2. 2026Discovering Differences in Strategic Behavior Between Humans and LLMscited by 3
  3. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
  4. 2026The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discoursecited by 2
  5. 2026Game Arena: Strategic LLM Evaluation in Competitive Environmentscited by 2
  6. 2026Evaluating Large Language Models in a Complex Hidden Role Gamecited by 1
  7. 2025LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluationcited by 0
  8. 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Playcited by 0
  9. 2026SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Gamecited by 0
  10. 2026When Reasoning Narrows the Move: Diversity Collapse in LLM Game Playcited by 0
  11. 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
  2. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  3. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games
  4. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models
  5. 2025GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
  6. 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
  7. 2025KORGym: A Dynamic Game Platform for LLM Reasoning Evaluation
  8. 2024TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMs