mlsanityData to 7 Oct 2026

Skill Check: Some Considerations on the Evaluation of Gamemastering Models for Role-playing Games

arXiv 2309.13702 ↗First posted 24 Sep 2023cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

In role-playing games a Game Master (GM) is the player in charge of the game, who must design the challenges the players face and narrate the outcomes of their actions. In this work we discuss some challenges to model GMs from an Interactive Storytelling and Natural Language Processing perspective. Following those challenges we propose three test categories to evaluate such dialogue systems, and we use them to test ChatGPT, Bard and OpenAssistant as out-of-the-box GMs.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 57 references that are in the map. First posted on arXiv 24 Sep 2023.

Builds on

11 of 57 references.

References · 11

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  3. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  4. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  5. 2014Narrative Planning: Balancing Plot and Charactercited by 61
  6. 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
  7. 2023CALYPSO: LLMs as Dungeon Masters' Assistantscited by 23
  8. 2023Designing Participatory AI: Creative Professionals' Worries and Expectations about Generative AIcited by 20
  9. 2023FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Informationcited by 18
  10. 2018Machine Common Sense Concept Papercited by 17
  11. 2021A guided journey through non-interactive automatic story generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025PAYADOR: A Minimalist Approach to Grounding Language Models on Structured Data for Interactive Storytelling and Role-playing Gamescited by 2
  2. 2026World-State Transformations for Neuro-symbolic Interactive Storytellingcited by 2
  3. 2026IVIE: A Neuro-symbolic Approach to Incremental and Validated Generation of Interactive Fiction Worldscited by 0