mlsanityData to 14 Oct 2026

Visual Storytelling

arXiv 1604.03968 ↗First posted 13 Apr 2016cs.CL cs.AI cs.CVcited by 176

Not on the map. Why?

Cited by 29 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce the first dataset for sequential vision-to-language, and explore how this data may be used for the task of visual storytelling. The first release of this dataset, SIND v.1, includes 81,743 unique photos in 20,211 sequences, aligned to both descriptive (caption) and story language. We establish several strong baselines for the storytelling task, and motivate an automatic metric to benchmark progress. Modelling concrete description as well as figurative and social language, as provided in this dataset and the storytelling task, has the potential to move artificial intelligence from basic understandings of typical visual scenes towards more and more human-like understanding of grounded event structure and subjective expression.

Key facts. Cited by 176 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 28; 2023: 42; 2024: 41; 2025: 33; 2026: 11.

176 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  3. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  4. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  5. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  6. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
  7. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  8. 2023Generating Images with Multimodal Language Modelscited by 260
  9. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  10. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  11. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 185
  12. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
38 more
  1. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 131
  2. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 127
  3. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  4. 2021Screen2Words: Automatic Mobile UI Summarization with Multimodal Learningcited by 120
  5. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
  6. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  7. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 105
  8. 2025One-Minute Video Generation with Test-Time Trainingcited by 101
  9. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  10. 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Modelscited by 77
  11. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  12. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 75
  13. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 75
  14. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  15. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 73
  16. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
  17. 2022Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Modelscited by 70
  18. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 69
  19. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 53
  20. 2023AutoAD: Movie Description in Contextcited by 50
  21. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 46
  22. 2023CLAIR: Evaluating Image Captions with Large Language Modelscited by 43
  23. 2021Weakly Supervised Temporal Adjacent Network for Language Groundingcited by 37
  24. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  25. 2024MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Modelscited by 33
  26. 2023Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequencescited by 31
  27. 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generationcited by 31
  28. 2022Multimodal Knowledge Alignment with Reinforcement Learningcited by 30
  29. 2022Visualize Before You Write: Imagination-Guided Open-Ended Text Generationcited by 30
  30. 2021Change Detection Meets Visual Question Answeringcited by 29
  31. 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 29
  32. 2022Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Taskscited by 28
  33. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  34. 2024Holistic Evaluation for Interleaved Text-and-Image Generationcited by 28
  35. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  36. 2023StoryGPT-V: Large Language Models as Consistent Story Visualizerscited by 25
  37. 2021Focused Attention Improves Document-Grounded Generationcited by 24
  38. 2024MIO: A Foundation Model on Multimodal Tokenscited by 24

50 most cited of 176 All 176 (JSON)