LUDOBENCH: Evaluating LLM Behavioural Decision-Making Through Spot-Based Board Game Scenarios in Ludo
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
We introduce LudoBench, a benchmark for evaluating LLM strategic reasoning in Ludo, a stochastic multi-agent board game whose dice mechanics, piece capture, safe-square navigation, and home-path progression introduce meaningful planning complexity. LudoBench comprises 480 handcrafted spot scenarios across 12 behaviorally distinct decision categories, each isolating a specific strategic choice. We additionally contribute a fully functional 4-player Ludo simulator supporting Random, Heuristic, Game-Theory, and LLM agents. The game-theory agent uses Expectiminimax search with depth-limited lookahead to provide a principled strategic ceiling beyond greedy heuristics. Evaluating six models spanning four model families, we find that all models agree with the game-theory baseline only 40-46% of the time. Models split into distinct behavioral archetypes: finishers that complete pieces but neglect development, and builders that develop but never finish. Each archetype captures only half of the game theory strategy. Models also display measurable behavioral shifts under history-conditioned grudge framing on identical board states, revealing prompt-sensitivity as a key vulnerability. LudoBench provides a lightweight and interpretable framework for benchmarking LLM strategic reasoning under uncertainty. All code, the spot dataset (480 entries) and model outputs are available at https://anonymous.4open.science/r/LudoBench-5CBF/
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 14 of its 31 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
14 of 31 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2023GPT-4 Technical Reportcited by 18,278
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2021Training Verifiers to Solve Math Word Problemscited by 9,916
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
- 2020Measuring Massive Multitask Language Understandingcited by 7,876
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,880
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 908
- 2024Understanding the planning of LLM agents: A surveycited by 416
- 2018TextWorld: A Learning Environment for Text-based Gamescited by 306
- 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 151
1 more
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
- 2025Approximating Human Strategic Reasoning with LLM-Enhanced Recursive Reasoners Leveraging Multi-agent Hypergames
- 2025Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play
- 2024TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMs
- 2025CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs
- 2026SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire
- 2023ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents
- 2025Tracing LLM Reasoning Processes with Strategic Games: A Framework for Planning, Revision, and Resource-Constrained Decision Making