mlsanityData to 7 Oct 2026

A Temporal Variational Model for Story Generation

arXiv 2109.06807 ↗First posted 14 Sep 2021cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Recent language models can generate interesting and grammatically correct text in story generation but often lack plot development and long-term coherence. This paper experiments with a latent vector planning approach based on a TD-VAE (Temporal Difference Variational Autoencoder), using the model for conditioning and reranking for text generation. The results demonstrate strong performance in automatic cloze and swapping evaluations. The human judgments show stories generated with TD-VAE reranking improve on a GPT-2 medium baseline and show comparable performance to a hierarchical LSTM reranking model. Conditioning on the latent vectors proves disappointing and deteriorates performance in human evaluation because it reduces the diversity of generation, and the models don't learn to progress the narrative. This highlights an important difference between technical task performance (e.g. cloze) and generating interesting stories.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

47 of 64 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  10. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
35 more
  1. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  4. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  5. 2018Universal Sentence Encodercited by 485
  6. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  7. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  8. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  9. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  10. 2015Skip-Thought Vectorscited by 321
  11. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  12. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  13. 2018An efficient framework for learning sentence representationscited by 197
  14. 2016A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialoguescited by 174
  15. 2019Strategies for Structuring Story Generationcited by 124
  16. 2018Learning to Write with Cooperative Discriminatorscited by 119
  17. 2020Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Spacecited by 114
  18. 2019Semantics-aware BERT for Language Understandingcited by 111
  19. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  20. 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
  21. 2019Do Massively Pretrained Language Models Make Better Storytellers?cited by 90
  22. 2018Probabilistic Recurrent State-Space Modelscited by 64
  23. 2017Event Representations for Automated Story Generation with Deep Neural Netscited by 62
  24. 2021OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metricscited by 61
  25. 2020UNION: An Unreferenced Metric for Evaluating Open-ended Story Generationcited by 60
  26. 2018Temporal Difference Variational Auto-Encodercited by 48
  27. 2019Plan, Write, and Revise: an Interactive System for Open-Domain Story Generationcited by 43
  28. 2018A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generationcited by 42
  29. 2017The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Taskcited by 39
  30. 2019Variational Temporal Abstractioncited by 37
  31. 2019Story Realization: Expanding Plot Events into Sentencescited by 35
  32. 2019Encoder-Agnostic Adaptation for Conditional Language Generationcited by 27
  33. 2013New Alignment Methods for Discriminative Book Summarizationcited by 18
  34. 2020Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representationcited by 13
  35. 2019The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summariescited by 8

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 2.

5 papers.

  1. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  2. 2024Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluationcited by 32
  3. 2024SWAG: Storytelling With Action Guidancecited by 8
  4. 2023Conditional Generative Modeling for High-dimensional Marked Temporal Point Processescited by 7
  5. 2022Topical Segmentation of Spoken Narratives: A Test Case on Holocaust Survivor Testimoniescited by 4