mlsanityData to 7 Oct 2026

Does Reasoning Help LLM Agents Play Dungeons and Dragons? A Prompt Engineering Experiment

arXiv 2510.18112 ↗First posted 20 Oct 2025cs.CLcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

This paper explores the application of Large Language Models (LLMs) and reasoning to predict Dungeons & Dragons (DnD) player actions and format them as Avrae Discord bot commands. Using the FIREBALL dataset, we evaluated a reasoning model, DeepSeek-R1-Distill-LLaMA-8B, and an instruct model, LLaMA-3.1-8B-Instruct, for command generation. Our findings highlight the importance of providing specific instructions to models, that even single sentence changes in prompts can greatly affect the output of models, and that instruct models are sufficient for this task compared to reasoning models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 8 of its 25 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

8 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Reasoning and the Trusting Behavior of DeepSeek and GPT: An Experiment Revealing Hidden Fault Lines in Large Language Models18 Feb
  2. 2024OpenAI o1 System Card21 Dec

Other references · 6

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  5. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  6. 2025Enhancing Generalization in Chain of Thought Reasoning for Smaller Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GameArena: Evaluating LLM Reasoning through Live Computer Games
  2. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  3. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games
  4. 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents
  5. 2025Improving LLM Reasoning through Interpretable Role-Playing Steering
  6. 2023AvalonBench: Evaluating LLMs Playing the Game of Avalon
  7. 2025First Steps Towards Overhearing LLM Agents: A Case Study With Dungeons & Dragons Gameplay
  8. 2026How reliable are LLMs when it comes to playing dice?