mlsanityData to 7 Oct 2026

Controllable Neural Story Plot Generation via Reward Shaping

arXiv 1809.10736 ↗First posted 27 Sep 2018cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

Language-modeling--based approaches to story plot generation attempt to construct a plot by sampling from a language model (LM) to predict the next character, word, or sentence to add to the story. LM techniques lack the ability to receive guidance from the user to achieve a specific goal, resulting in stories that don't have a clear sense of progression and lack coherence. We present a reward-shaping technique that analyzes a story corpus and produces intermediate rewards that are backpropagated into a pre-trained LM in order to guide the model towards a given goal. Automated evaluations show our technique can create a model that generates story plots which consistently achieve a specified goal. Human-subject studies show that the generated stories have more plausible event ordering than baseline plot generation techniques.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 18; 2023: 9; 2024: 6; 2025: 3; 2026: 1.

49 papers.

Sort:

By area:

  1. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  2. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
  3. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  4. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  5. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  6. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  7. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 81
  8. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  9. 2022Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summariescited by 47
  10. 2021Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairscited by 45
  11. 2021Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoningcited by 29
  12. 2021Automatic Story Generation: Challenges and Attemptscited by 28
37 more
  1. 2021Creativity and Machine Learning: A Surveycited by 21
  2. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  3. 2021GraphPlan: Story Generation by Planning with Event Graphcited by 18
  4. 2023Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approachcited by 17
  5. 2022Go Back in Time: Generating Flashbacks in Stories with Event Temporal Promptscited by 15
  6. 2021Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learningcited by 14
  7. 2022Robust Preference Learning for Storytelling via Contrastive Reinforcement Learningcited by 14
  8. 2023GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidencecited by 14
  9. 2021Telling Stories through Multi-User Dialogue by Modeling Character Relationscited by 13
  10. 2023LongStory: Coherent, Complete and Length Controlled Long story Generationcited by 12
  11. 2021Energy-Based Models for Code Generation under Compilability Constraintscited by 10
  12. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  13. 2023SuperHF: Supervised Iterative Learning from Human Feedbackcited by 10
  14. 2022TropeTwist: Trope-based Narrative Structure Generationcited by 8
  15. 2024SWAG: Storytelling With Action Guidancecited by 8
  16. 2021Changing the Mind of Transformers for Topically-Controllable Language Generationcited by 7
  17. 2021Fabula Entropy Indexing: Objective Measures of Story Coherencecited by 7
  18. 2022CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understandingcited by 7
  19. 2024Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMscited by 7
  20. 2024Creativity in AI: Progresses and Challengescited by 7
  21. 2021Gradient-Based Mixed Planning with Symbolic and Numeric Action Parameterscited by 6
  22. 2022Incorporating Commonsense Knowledge into Story Ending Generation via Heterogeneous Graph Networkscited by 6
  23. 2022CHAE: Fine-Grained Controllable Story Generation with Characters, Actions and Emotionscited by 6
  24. 2022Story Designer: Towards a Mixed-Initiative Tool to Create Narrative Structurescited by 6
  25. 2023Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Controlcited by 5
  26. 2022Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledgecited by 3
  27. 2025COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Modelscited by 3
  28. 2021TopNet: Learning from Neural Topic Model to Generate Long Storiescited by 2
  29. 2022Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learningcited by 2
  30. 2022Psychology-guided Controllable Story Generationcited by 2
  31. 2025Preference-Guided Learning for Sparse-Reward Multi-Agent Reinforcement Learningcited by 2
  32. 2023Visual Story Generation Based on Emotion and Keywordscited by 1
  33. 2025Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generationcited by 1
  34. 2022Computational Storytelling and Emotions: A Surveycited by 0
  35. 2024Reinforcement Learning from Bagged Rewardcited by 0
  36. 2024CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language Generationscited by 0
  37. 2026StorySpark: Module-wise Evolutionary Search for Story Premise Generationcited by 0