# Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization

arXiv: https://arxiv.org/abs/2502.07096 · HTML: https://mlsanity.com/paper/2502.07096/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [Text-based Editing of Talking-head Video](https://mlsanity.com/paper/1906.01524/) (2019)
- [Video Summarization with Attention-Based Encoder-Decoder Networks](https://mlsanity.com/paper/1708.09545/) (2017)
- [Text Summarization Techniques: A Brief Survey](https://mlsanity.com/paper/1707.02268/) (2017)
- [TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models](https://mlsanity.com/paper/2308.14430/) (2023)
- [LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing](https://mlsanity.com/paper/2402.10294/) (2024)
- [Hierarchical Recurrent Neural Network for Video Summarization](https://mlsanity.com/paper/1904.12251/) (2019)
- [Multimodal Abstractive Summarization for How2 Videos](https://mlsanity.com/paper/1906.07901/) (2019)
- [TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch](https://mlsanity.com/paper/2310.17864/) (2023)
- [Rescribe: Authoring and Automatically Editing Audio Descriptions](https://mlsanity.com/paper/2010.03667/) (2020)
- [SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description](https://mlsanity.com/paper/2408.13608/) (2024)
- [AVscript: Accessible Video Editing with Audio-Visual Scripts](https://mlsanity.com/paper/2302.14117/) (2023)
- [ReelFramer: Human-AI Co-Creation for News-to-Video Translation](https://mlsanity.com/paper/2304.09653/) (2023)
- [B-Script: Transcript-based B-roll Video Editing with Recommendations](https://mlsanity.com/paper/1902.11216/) (2019)
- [ExpressEdit: Video Editing with Natural Language and Sketching](https://mlsanity.com/paper/2403.17693/) (2024)
- [EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control](https://mlsanity.com/paper/2410.00316/) (2024)
- [PodReels: Human-AI Co-Creation of Video Podcast Teasers](https://mlsanity.com/paper/2311.05867/) (2023)
- [Hierarchical Summarization for Longform Spoken Dialog](https://mlsanity.com/paper/2108.09597/) (2021)
- [Video Summarization Overview](https://mlsanity.com/paper/2210.11707/) (2022)
- [ElasticPlay: Interactive Video Summarization with Dynamic Time Budgets](https://mlsanity.com/paper/1708.06858/) (2017)

## Built on by (most cited first, 6 of 6)

- [From Long Videos to Engaging Clips: A Human-Inspired Video Editing Framework with Multimodal Narrative Understanding](https://mlsanity.com/paper/2507.02790/) (2025)
- [A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation](https://mlsanity.com/paper/2604.10456/) (2026)
- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing](https://mlsanity.com/paper/2604.05076/) (2026)
- [StreamWise: Serving Multi-Modal Generation in Real-Time at Scale](https://mlsanity.com/paper/2603.05800/) (2026)
- [Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights](https://mlsanity.com/paper/2609.20768/) (2026)

## Abstract (arXiv)

Short-form videos are popular on platforms like TikTok and Instagram as they quickly capture viewers' attention. Many creators repurpose their long-form videos to produce short-form videos, but creators report that planning, extracting, and arranging clips from long-form videos is challenging. Currently, creators make extractive short-form videos composed of existing long-form video clips or abstractive short-form videos by adding newly recorded narration to visuals. While extractive videos maintain the original connection between audio and visuals, abstractive videos offer flexibility in selecting content to be included in a shorter time. We present Lotus, a system that combines both approaches to balance preserving the original content with flexibility over the content. Lotus first creates an abstractive short-form video by generating both a short-form script and its corresponding speech, then matching long-form video clips to the generated narration. Creators can then add extractive clips with an automated method or Lotus's editing interface. Lotus's interface can be used to further refine the short-form video. We compare short-form videos generated by Lotus with those using an extractive baseline method. In our user study, we compare creating short-form videos using Lotus to participants' existing practice.