# Reinforcement Learning Environment with LLM-Controlled Adversary in D&D 5th Edition Combat

arXiv: https://arxiv.org/abs/2503.15726 · HTML: https://mlsanity.com/paper/2503.15726/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Guiding Pretraining in Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2302.06692/) (2023)
- [Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions](https://mlsanity.com/paper/2304.02868/) (2023)
- [FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information](https://mlsanity.com/paper/2305.01528/) (2023)
- [Reinforcement Learning In Two Player Zero Sum Simultaneous Action Games](https://mlsanity.com/paper/2110.04835/) (2021)
- [Re-conceptualising the Language Game Paradigm in the Framework of Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2004.04722/) (2020)

## Built on by (most cited first, 3 of 3)

- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPG](https://mlsanity.com/paper/2607.02802/) (2026)
- [DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat](https://mlsanity.com/paper/2607.29577/) (2026)

## Abstract (arXiv)

The objective of this study is to design and implement a reinforcement learning (RL) environment using D&D 5E combat scenarios to challenge smaller RL agents through interaction with a robust adversarial agent controlled by advanced Large Language Models (LLMs) like GPT-4o and LLaMA 3 8B. This research employs Deep Q-Networks (DQN) for the smaller agents, creating a testbed for strategic AI development that also serves as an educational tool by simulating dynamic and unpredictable combat scenarios. We successfully integrated sophisticated language models into the RL framework, enhancing strategic decision-making processes. Our results indicate that while RL agents generally outperform LLM-controlled adversaries in standard metrics, the strategic depth provided by LLMs significantly enhances the overall AI capabilities in this complex, rule-based setting. The novelty of our approach and its implications for mastering intricate environments and developing adaptive strategies are discussed, alongside potential innovations in AI-driven interactive simulations. This paper aims to demonstrate how integrating LLMs can create more robust and adaptable AI systems, providing valuable insights for further research and educational applications.