mlsanityData to 7 Oct 2026

Plot and Rework: Modeling Storylines for Visual Storytelling

arXiv 2105.06950 ↗First posted 14 May 2021cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Writing a coherent and engaging story is not easy. Creative writers use their knowledge and worldview to put disjointed elements together to form a coherent storyline, and work and rework iteratively toward perfection. Automated visual storytelling (VIST) models, however, make poor use of external knowledge and iterative generation when attempting to create stories. This paper introduces PR-VIST, a framework that represents the input image sequence as a story graph in which it finds the best path to form a storyline. PR-VIST then takes this path and learns to generate the final story via an iterative training process. This framework produces stories that are superior in terms of diversity, coherence, and humanness, per both automatic and human evaluations. An ablation study shows that both plotting and reworking contribute to the model's superiority.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 14 May 2021.

Builds on

28 of 41 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  7. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  8. 2018Image Generation from Scene Graphscited by 324
  9. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  10. 2016Visual Storytellingcited by 176
  11. 2019Strategies for Structuring Story Generationcited by 124
  12. 2017Towards Diverse and Natural Image Descriptions via a Conditional GANcited by 59
16 more
  1. 2018No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytellingcited by 46
  2. 2019UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answeringcited by 37
  3. 2017Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffoldcited by 33
  4. 2019What Makes A Good Story? Designing Composite Rewards for Visual Storytellingcited by 29
  5. 2019Knowledge-Enriched Visual Storytellingcited by 26
  6. 2018GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generationcited by 23
  7. 2018Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generationcited by 21
  8. 2018Contextualize, Show and Tell: A Neural Visual Storytellercited by 13
  9. 2020Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytellingcited by 13
  10. 2019Visual Story Post-Editingcited by 10
  11. 2020Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytellingcited by 8
  12. 2019The Steep Road to Happily Ever After: An Analysis of Current Visual Storytelling Modelscited by 6
  13. 2017Multimodal Storytelling via Generative Adversarial Imitation Learningcited by 3
  14. 2018Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytellingcited by 3
  15. 2019Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communicationcited by 3
  16. 2018Knowledge Base Relation Detection via Multi-View Matchingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 8; 2025: 3; 2026: 1.

14 papers.

Sort:

  1. 2025VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?cited by 11
  2. 2023Detecting and Grounding Important Characters in Visual Storiescited by 6
  3. 2024Generating Coherent Sequences of Visual Illustrations for Real-World Manual Taskscited by 6
  4. 2023Visual Storytelling with Question-Answer Planscited by 5
  5. 2024SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytellingcited by 5
  6. 2024Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetitioncited by 5
  7. 2024Story Generation from Visual Inputs: Techniques, Related Tasks, and Challengescited by 3
  8. 2024Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translationcited by 2
  9. 2024Generating Visual Stories with Grounded and Coreferent Characterscited by 1
  10. 2025MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Modelcited by 1
  11. 2024TARN-VIST: Topic Aware Reinforcement Network for Visual Storytellingcited by 0
  12. 2024A Pattern to Align Them All: Integrating Different Modalities to Define Multi-Modal Entitiescited by 0
2 more
  1. 2025From Image Captioning to Visual Storytellingcited by 0
  2. 2026StoryMovie: A Dataset for Semantic Alignment of Visual Stories with Movie Scripts and Subtitlescited by 0