mlsanityData to 7 Oct 2026

FairyTailor: A Multimodal Generative Framework for Storytelling

arXiv 2108.04324 ↗First posted 13 Jul 2021cs.CL cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Storytelling is an open-ended task that entails creative thinking and requires a constant flow of ideas. Natural language generation (NLG) for storytelling is especially challenging because it requires the generated text to follow an overall theme while remaining creative and diverse to engage the reader. In this work, we introduce a system and a web-based demo, FairyTailor, for human-in-the-loop visual story co-creation. Users can create a cohesive children's fairytale by weaving generated texts and retrieved images with their input. FairyTailor adds another modality and modifies the text generation process to produce a coherent and creative sequence of text and images. To our knowledge, this is the first dynamic tool for multimodal story generation that allows interactive co-formation of both texts and images. It allows users to give feedback on co-created stories and share their results.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 27 of its 48 references that are in the map. First posted on arXiv 13 Jul 2021.

Builds on

27 of 48 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  12. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
15 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  2. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  6. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  7. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  8. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  9. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
  10. 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 137
  11. 2020STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generationcited by 56
  12. 2017Story Generation from Sequence of Independent Short Descriptionscited by 26
  13. 2020Narrative Interpolation for Generating and Understanding Storiescited by 19
  14. 2019Multimodal Generative Models for Compositional Representation Learningcited by 13
  15. 2020Schema-Guided Natural Language Generationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 3; 2026: 2.

9 papers.

  1. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  2. 2024Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 48
  3. 2024DataNarrative: Automated Data-Driven Storytelling with Visualizations and Textscited by 25
  4. 2023Toward Artificial Empathy for Human-Centered Design: A Frameworkcited by 6
  5. 2025Guiding Generative Storytelling with Knowledge Graphscited by 2
  6. 2025Influence-driven Curriculum Learning for Pre-training on Limited Datacited by 2
  7. 2022VScript: Controllable Script Generation with Visual Presentationcited by 1
  8. 2026DATAREEL: Automated Data-Driven Video Story Generation with Animationscited by 1
  9. 2026Does Episodic Memory Help Close the Lexical Frequency Gap in Sensitivity to Syntactic Contrasts? A Test Using Retrieval-Augmented Language Modelscited by 0