mlsanityData to 7 Oct 2026

Story Shaping: Teaching Agents Human-like Behavior with Stories

arXiv 2301.10107 ↗First posted 24 Jan 2023cs.AI cs.GT cs.HCcited by 7

Not on the map. Why?

Abstract (arXiv)

Reward design for reinforcement learning agents can be difficult in situations where one not only wants the agent to achieve some effect in the world but where one also cares about how that effect is achieved. For example, we might wish for an agent to adhere to a tacit understanding of commonsense, align itself to a preference for how to behave for purposes of safety, or taking on a particular role in an interactive game. Storytelling is a mode for communicating tacit procedural knowledge. We introduce a technique, Story Shaping, in which a reinforcement learning agent infers tacit knowledge from an exemplar story of how to accomplish a task and intrinsically rewards itself for performing actions that make its current environment adhere to that of the inferred story world. Specifically, Story Shaping infers a knowledge graph representation of the world state from observations, and also infers a knowledge graph from the exemplar story. An intrinsic reward is generated based on the similarity between the agent's inferred world state graph and the inferred story world graph. We conducted experiments in text-based games requiring commonsense reasoning and shaping the behaviors of agents as virtual game characters.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 24 Jan 2023.

Builds on

32 of 51 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Graph Attention Networkscited by 8,854
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  8. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  9. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  10. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  11. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  12. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
20 more
  1. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  2. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  3. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  4. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  5. 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
  6. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  7. 2018Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learningcited by 53
  8. 2020How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worldscited by 38
  9. 2020How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worldscited by 35
  10. 2022Aligning to Social Norms and Values in Interactive Narrativescited by 32
  11. 2021Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoningcited by 29
  12. 2020Bringing Stories Alive: Generating Interactive Fiction Worldscited by 28
  13. 2018Hierarchical Reinforcement Learning with Hindsightcited by 26
  14. 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
  15. 2020Playing Text-Based Games with Common Sensecited by 21
  16. 2019Learning Norms from Stories: A Prior for Value Aligned Agentscited by 19
  17. 2021Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codescited by 19
  18. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  19. 2021Training Value-Aligned Reinforcement Learning Agents Using a Normative Priorcited by 14
  20. 2021Guiding Neural Story Generation with Reader Modelscited by 12

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 3.

7 papers.

  1. 2025TALES: Text Adventure Learning Environment Suitecited by 9
  2. 2023Dialogue Shaping: Empowering Agents through NPC Interactioncited by 8
  3. 2023Thespian: Multi-Character Text Role-Playing Game Agentscited by 6
  4. 2023Ambient Adventures: Teaching ChatGPT on Developing Complex Storiescited by 5
  5. 2024A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worldscited by 2
  6. 2025Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challengescited by 1
  7. 2025ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Learning Knowledge Graph-based World Models of Textual Environments
  2. 2025Narrative-Guided Reinforcement Learning: A Platform for Studying Language Model Influence on Decision Making
  3. 2026Rewarding Creativity: A Human-Aligned Generative Reward Model for Reinforcement Learning in Storytelling
  4. 2021Modeling Worlds in Text
  5. 2026Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling
  6. 2025The Power of Stories: Narrative Priming Shapes How LLM Agents Collaborate and Compete
  7. 2023Knowledge-enhanced Agents for Interactive Text Games
  8. 2026When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulations