# ScreenWriter: Automatic Screenplay Generation and Movie Summarisation

arXiv: https://arxiv.org/abs/2410.19809 · HTML: https://mlsanity.com/paper/2410.19809/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://mlsanity.com/paper/2305.03726/) (2023)
- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [AlignScore: Evaluating Factual Consistency with a Unified Alignment Function](https://mlsanity.com/paper/2305.16739/) (2023)
- [MovieNet: A Holistic Dataset for Movie Understanding](https://mlsanity.com/paper/2007.10937/) (2020)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning](https://mlsanity.com/paper/2111.13196/) (2021)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading](https://mlsanity.com/paper/2310.05029/) (2023)
- [Spatio-Temporal Graph for Video Captioning with Knowledge Distillation](https://mlsanity.com/paper/2003.13942/) (2020)
- [LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization](https://mlsanity.com/paper/2301.13298/) (2023)
- [MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning](https://mlsanity.com/paper/2005.05402/) (2020)
- [Open-book Video Captioning with Retrieve-Copy-Generate Network](https://mlsanity.com/paper/2103.05284/) (2021)
- [Long Document Summarization with Top-down and Bottom-up Inference](https://mlsanity.com/paper/2203.07586/) (2022)
- [AutoAD III: The Prequel -- Back to the Pixels](https://mlsanity.com/paper/2404.14412/) (2024)
- [Movie Plot Analysis via Turning Point Identification](https://mlsanity.com/paper/1908.10328/) (2019)
- [Hierarchical Modular Network for Video Captioning](https://mlsanity.com/paper/2111.12476/) (2021)
- [Movie Summarization via Sparse Graph Construction](https://mlsanity.com/paper/2012.07536/) (2020)
- [A Modular Approach for Multimodal Summarization of TV Shows](https://mlsanity.com/paper/2403.03823/) (2024)
- [Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition](https://mlsanity.com/paper/2111.08774/) (2021)
- [Hierarchical3D Adapters for Long Video-to-text Summarization](https://mlsanity.com/paper/2210.04829/) (2022)
- [Movies2Scenes: Using Movie Metadata to Learn Scene Representation](https://mlsanity.com/paper/2202.10650/) (2022)

## Built on by (most cited first, 5 of 5)

- [Generative AI for Film Creation: A Survey of Recent Advances](https://mlsanity.com/paper/2504.08296/) (2025)
- [CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generation](https://mlsanity.com/paper/2510.06231/) (2025)
- [Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration](https://mlsanity.com/paper/2605.17423/) (2026)
- [REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing](https://mlsanity.com/paper/2505.18880/) (2025)
- [S^2tory: Story Spine Distillation for Movie Script Summarization](https://mlsanity.com/paper/2605.03244/) (2026)

## Abstract (arXiv)

The proliferation of creative video content has driven demand for textual descriptions or summaries that allow users to recall key plot points or get an overview without watching. The volume of movie content and speed of turnover motivates automatic summarisation, which is nevertheless challenging, requiring identifying character intentions and very long-range temporal dependencies. The few existing methods attempting this task rely heavily on textual screenplays as input, greatly limiting their applicability. In this work, we propose the task of automatic screenplay generation, and a method, ScreenWriter, that operates only on video and produces output which includes dialogue, speaker names, scene breaks, and visual descriptions. ScreenWriter introduces a novel algorithm to segment the video into scenes based on the sequence of visual vectors, and a novel method for the challenging problem of determining character names, based on a database of actors' faces. We further demonstrate how these automatic screenplays can be used to generate plot synopses with a hierarchical summarisation method based on scene breaks. We test the quality of the final summaries on the recent MovieSum dataset, which we augment with videos, and show that they are superior to a number of comparison models which assume access to goldstandard screenplays.