# Grammar and Gameplay-aligned RL for Game Description Generation with LLMs

arXiv: https://arxiv.org/abs/2503.15783 · HTML: https://mlsanity.com/paper/2503.15783/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://mlsanity.com/paper/2401.08967/) (2024)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Level Generation Through Large Language Models](https://mlsanity.com/paper/2302.05817/) (2023)
- [MarioGPT: Open-Ended Text2Level Generation through Large Language Models](https://mlsanity.com/paper/2302.05981/) (2023)
- [Ludii -- The Ludemic General Game System](https://mlsanity.com/paper/1905.05013/) (2019)
- [Game Generation via Large Language Models](https://mlsanity.com/paper/2404.08706/) (2024)
- [GAVEL: Generating Games Via Evolution and Language Models](https://mlsanity.com/paper/2407.09388/) (2024)
- [Practical PCG Through Large Language Models](https://mlsanity.com/paper/2305.18243/) (2023)
- [General Video Game Rule Generation](https://mlsanity.com/paper/1906.05160/) (2019)
- [DreamCraft: Text-Guided Generation of Functional 3D Environments in Minecraft](https://mlsanity.com/paper/2404.15538/) (2024)
- [Unbounded: A Generative Infinite Game of Character Life Simulation](https://mlsanity.com/paper/2410.18975/) (2024)
- [General Board Game Concepts](https://mlsanity.com/paper/2107.01078/) (2021)
- [General Game Heuristic Prediction Based on Ludeme Descriptions](https://mlsanity.com/paper/2105.12846/) (2021)
- [Grammar-based Game Description Generation using Large Language Models](https://mlsanity.com/paper/2407.17404/) (2024)
- [Measuring Board Game Distance](https://mlsanity.com/paper/2301.03913/) (2023)
- [Adversarial Random Forest Classifier for Automated Game Design](https://mlsanity.com/paper/2107.12501/) (2021)
- [The Ludii Game Description Language is Universal](https://mlsanity.com/paper/2205.00451/) (2022)
- [Automatic Generation of Board Game Manuals](https://mlsanity.com/paper/2109.09507/) (2021)

## Built on by (most cited first, 2 of 2)

- [Divide-Fuse-Conquer: Eliciting "Aha Moments" in Multi-Scenario Games](https://mlsanity.com/paper/2505.16401/) (2025)
- [From Gameplay Traces to Game Mechanics: Causal Induction with Large Language Models](https://mlsanity.com/paper/2602.00190/) (2026)

## Abstract (arXiv)

Game Description Generation (GDG) is the task of generating a game description written in a Game Description Language (GDL) from natural language text. Previous studies have explored generation methods leveraging the contextual understanding capabilities of Large Language Models (LLMs); however, accurately reproducing the game features of the game descriptions remains a challenge. In this paper, we propose reinforcement learning-based fine-tuning of LLMs for GDG (RLGDG). Our training method simultaneously improves grammatical correctness and fidelity to game concepts by introducing both grammar rewards and concept rewards. Furthermore, we adopt a two-stage training strategy where Reinforcement Learning (RL) is applied following Supervised Fine-Tuning (SFT). Experimental results demonstrate that our proposed method significantly outperforms baseline methods using SFT alone. Our code is available at https://github.com/tsunehiko/rlgdg