# Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling

arXiv: https://arxiv.org/abs/2601.17226 · HTML: https://mlsanity.com/paper/2601.17226/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 18 of 18)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Can language models learn from explanations in context?](https://mlsanity.com/paper/2204.02329/) (2022)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Counterfactual Story Reasoning and Generation](https://mlsanity.com/paper/1909.04076/) (2019)
- [Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation](https://mlsanity.com/paper/2208.11646/) (2022)
- [Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning](https://mlsanity.com/paper/2010.05906/) (2020)
- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text](https://mlsanity.com/paper/2410.04265/) (2024)
- [M-Prometheus: A Suite of Open Multilingual LLM Judges](https://mlsanity.com/paper/2504.04953/) (2025)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Cause and Effect: Can Large Language Models Truly Understand Causality?](https://mlsanity.com/paper/2402.18139/) (2024)
- [The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code](https://mlsanity.com/paper/2305.19213/) (2023)
- [CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans](https://mlsanity.com/paper/2406.15823/) (2024)
- [Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards](https://mlsanity.com/paper/2508.21476/) (2025)
- [Sketch and Customize: A Counterfactual Story Generator](https://mlsanity.com/paper/2104.00929/) (2021)
- [The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing](https://mlsanity.com/paper/2506.03310/) (2025)

## Abstract (arXiv)

Counterfactual storytelling exposes LLM shortcomings in causal reasoning, where few-shot prompting and SFT fail to produce logical and rational narratives. This case study introduces Retell, Reward, Repeat (RRR), a narrative theory-informed RL pipeline that teaches causal story structure. We extend an existing counterfactual stories dataset with human-annotated stages of narrative equilibrium, synthesising Structuralist Narratology with scalar narrativity. An aligned LLM-as-judge automates this framework to output reward signals, teaching the policy LLM to generate texts with causal narrative structure. Evaluations show that RRR-trained 7/8B LLMs outperform few-shot and SFT baselines in logic, rationality, and completeness, also aligning with a blind human preference subset. RRR offers a linguistically grounded, cost-effective post-training mechanism for LLM storytelling using a small, query-only dataset, showing the value of integrating linguistic theories into contemporary NLP.