Mastering Da Vinci Code: A Comparative Study of Transformer, LLM, and PPO-based Agents
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
The Da Vinci Code, a game of logical deduction and imperfect information, presents unique challenges for artificial intelligence, demanding nuanced reasoning beyond simple pattern recognition. This paper investigates the efficacy of various AI paradigms in mastering this game. We develop and evaluate three distinct agent architectures: a Transformer-based baseline model with limited historical context, several Large Language Model (LLM) agents (including Gemini, DeepSeek, and GPT variants) guided by structured prompts, and an agent based on Proximal Policy Optimization (PPO) employing a Transformer encoder for comprehensive game history processing. Performance is benchmarked against the baseline, with the PPO-based agent demonstrating superior win rates (58.5% ± 1.0%), significantly outperforming the LLM counterparts. Our analysis highlights the strengths of deep reinforcement learning in policy refinement for complex deductive tasks, particularly in learning implicit strategies from self-play. We also examine the capabilities and inherent limitations of current LLMs in maintaining strict logical consistency and strategic depth over extended gameplay, despite sophisticated prompting. This study contributes to the broader understanding of AI in recreational games involving hidden information and multi-step logical reasoning, offering insights into effective agent design and the comparative advantages of different AI approaches.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 7 of its 13 references that are in the map. First posted on arXiv 15 Jun 2025.
Builds on
7 of 13 references.
References · 7
- 2017Attention Is All You Needcited by 60,361
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Trust Region Policy Optimizationcited by 2,580
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
- 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 58
- 2024Mastering the Game of Guandan with Deep Reinforcement Learning and Behavior Regulatingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CAST: Game Solvers as Turn-Level Teachers for LLM Agents
- 2025Vox Deorum: A Hybrid LLM Architecture for 4X / Grand Strategy Game AI -- Lessons from Civilization V
- 2025Learning Game-Playing Agents with Generative Code Optimization
- 2026Clueing up LLMs with Tool-Augmented Deductive Reasoning
- 2022Multi-Game Decision Transformers
- 2025Policy-Conditioned Policies for Multi-Agent Task Solving
- 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization
- 2026Nemobot Games: Crafting Strategic AI Gaming Agents for Interactive Learning with Large Language Models