# TeaserGen: Generating Teasers for Long Documentaries

arXiv: https://arxiv.org/abs/2410.05586 · HTML: https://mlsanity.com/paper/2410.05586/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [MovieNet: A Holistic Dataset for Movie Understanding](https://mlsanity.com/paper/2007.10937/) (2020)
- [MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions](https://mlsanity.com/paper/2112.00431/) (2021)
- [CLIP-It! Language-Guided Video Summarization](https://mlsanity.com/paper/2107.00650/) (2021)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [VideoXum: Cross-modal Visual and Textural Summarization of Videos](https://mlsanity.com/paper/2303.12060/) (2023)
- [Scaling Up Video Summarization Pretraining with Large Language Models](https://mlsanity.com/paper/2404.03398/) (2024)
- [MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos](https://mlsanity.com/paper/2306.04216/) (2023)
- [PodReels: Human-AI Co-Creation of Video Podcast Teasers](https://mlsanity.com/paper/2311.05867/) (2023)
- [Towards Automated Movie Trailer Generation](https://mlsanity.com/paper/2404.03477/) (2024)
- [Cross-modal Common Representation Learning by Hybrid Transfer Network](https://mlsanity.com/paper/1706.00153/) (2017)
- [Transitive Hashing Network for Heterogeneous Multimedia Retrieval](https://mlsanity.com/paper/1608.04307/) (2016)

## Built on by (most cited first, 4 of 4)

- [VideoAgent: All-in-One Framework for Video Understanding and Editing](https://mlsanity.com/paper/2606.23327/) (2026)
- [Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity](https://mlsanity.com/paper/2604.04953/) (2026)
- [GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing](https://mlsanity.com/paper/2604.05076/) (2026)
- [REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing](https://mlsanity.com/paper/2505.18880/) (2025)

## Abstract (arXiv)

Teasers are an effective tool for promoting content in entertainment, commercial and educational fields. However, creating an effective teaser for long videos is challenging for it requires long-range multimodal modeling on the input videos, while necessitating maintaining audiovisual alignments, managing scene changes and preserving factual accuracy for the output teasers. Due to the lack of a publicly-available dataset, progress along this research direction has been hindered. In this work, we present DocumentaryNet, a collection of 1,269 documentaries paired with their teasers, featuring multimodal data streams of video, speech, music, sound effects and narrations. With DocumentaryNet, we propose a new two-stage system for generating teasers from long documentaries. The proposed TeaserGen system first generates the teaser narration from the transcribed narration of the documentary using a pretrained large language model, and then selects the most relevant visual content to accompany the generated narration through language-vision models. For narration-video matching, we explore two approaches: a pretraining-based model using pretrained contrastive language-vision models and a deep sequential model that learns the mapping between the narrations and visuals. Our experimental results show that the pretraining-based approach is more effective at identifying relevant visual content than directly trained deep autoregressive models.