# StoryMovie: A Dataset for Semantic Alignment of Visual Stories with Movie Scripts and Subtitles

arXiv: https://arxiv.org/abs/2602.21829 · HTML: https://mlsanity.com/paper/2602.21829/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [GROUNDHOG: Grounding Large Language Models to Holistic Segmentation](https://mlsanity.com/paper/2402.16846/) (2024)
- [Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences](https://mlsanity.com/paper/2301.08571/) (2023)
- [Plot and Rework: Modeling Storylines for Visual Storytelling](https://mlsanity.com/paper/2105.06950/) (2021)
- [GroundCap: A Visually Grounded Image Captioning Dataset](https://mlsanity.com/paper/2502.13898/) (2025)
- [Story Generation from Visual Inputs: Techniques, Related Tasks, and Challenges](https://mlsanity.com/paper/2406.02748/) (2024)
- [Character-aware audio-visual subtitling in context](https://mlsanity.com/paper/2410.11068/) (2024)
- [CHATTER: A Character Attribution Dataset for Narrative Understanding](https://mlsanity.com/paper/2411.05227/) (2024)

## Abstract (arXiv)

Visual storytelling models that correctly ground entities in images may still hallucinate semantic relationships, generating incorrect dialogue attribution, character interactions, or emotional states. We introduce StoryMovie, a dataset of 1,757 stories aligned with movie scripts and subtitles through LCS matching. Our alignment pipeline synchronizes screenplay dialogue with subtitle timestamps, enabling dialogue attribution by linking character names from scripts to temporal positions from subtitles. Using this aligned content, we generate stories that maintain visual grounding tags while incorporating authentic character names, dialogue, and relationship dynamics. We fine-tune Qwen Storyteller3 on this dataset, building on prior work in visual grounding and entity re-identification. Evaluation using DeepSeek V3 as judge shows that Storyteller3 achieves an 89.9% win rate against base Qwen2.5-VL 7B on subtitle alignment. Compared to Storyteller, trained without script grounding, Storyteller3 achieves 48.5% versus 38.0%, confirming that semantic alignment progressively improves dialogue attribution beyond visual grounding alone.