# Dialogue Shaping: Empowering Agents through NPC Interaction

arXiv: https://arxiv.org/abs/2307.15833 · HTML: https://mlsanity.com/paper/2307.15833/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 11 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 10 of 10)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [Keep CALM and Explore: Language Models for Action Generation in Text-based Games](https://mlsanity.com/paper/2010.02903/) (2020)
- [Graph Constrained Reinforcement Learning for Natural Language Action Spaces](https://mlsanity.com/paper/2001.08837/) (2020)
- [Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1812.01628/) (2018)
- [How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds](https://mlsanity.com/paper/2006.07409/) (2020)
- [Bringing Stories Alive: Generating Interactive Fiction Worlds](https://mlsanity.com/paper/2001.10161/) (2020)
- [Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games](https://mlsanity.com/paper/2010.11655/) (2020)
- [Inherently Explainable Reinforcement Learning in Natural Language](https://mlsanity.com/paper/2112.08907/) (2021)
- [Story Shaping: Teaching Agents Human-like Behavior with Stories](https://mlsanity.com/paper/2301.10107/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Teaching Large Language Models to Reason with Reinforcement Learning](https://mlsanity.com/paper/2403.04642/) (2024)
- [Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models](https://mlsanity.com/paper/2503.02623/) (2025)
- [Player-Driven Emergence in LLM-Driven Game Narrative](https://mlsanity.com/paper/2404.17027/) (2024)
- [Unbounded: A Generative Infinite Game of Character Life Simulation](https://mlsanity.com/paper/2410.18975/) (2024)
- [Branching Narratives: Character Decision Points Detection](https://mlsanity.com/paper/2405.07282/) (2024)
- [Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challenges](https://mlsanity.com/paper/2510.20641/) (2025)
- [Symbolically Scaffolded Play: Designing Role-Sensitive Prompts for Generative NPC Dialogue](https://mlsanity.com/paper/2510.25820/) (2025)

## Abstract (arXiv)

One major challenge in reinforcement learning (RL) is the large amount of steps for the RL agent needs to converge in the training process and learn the optimal policy, especially in text-based game environments where the action space is extensive. However, non-player characters (NPCs) sometimes hold some key information about the game, which can potentially help to train RL agents faster. Thus, this paper explores how to interact and converse with NPC agents to get the key information using large language models (LLMs), as well as incorporate this information to speed up RL agent's training using knowledge graphs (KGs) and Story Shaping.