Prompt-Driven Agentic Video Editing System: Autonomous Comprehension of Long-Form, Story-Driven Media
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Creators struggle to edit long-form, narrative-rich videos not because of UI complexity, but due to the cognitive demands of searching, storyboarding, and sequencing hours of footage. Existing transcript- or embedding-based methods fall short for creative workflows, as models struggle to track characters, infer motivations, and connect dispersed events. We present a prompt-driven, modular editing system that helps creators restructure multi-hour content through free-form prompts rather than timelines. At its core is a semantic indexing pipeline that builds a global narrative via temporal segmentation, guided memory compression, and cross-granularity fusion, producing interpretable traces of plot, dialogue, emotion, and context. Users receive cinematic edits while optionally refining transparent intermediate outputs. Evaluated on 400+ videos with expert ratings, QA, and preference studies, our system scales prompt-driven editing, preserves narrative coherence, and balances automation with creator control.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 21 of its 43 references that are in the map. First posted on arXiv 20 Sep 2025.
Builds on
21 of 43 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
- 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
- 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
- 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
- 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
- 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
8 more
- 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 183
- 2019Text-based Editing of Talking-head Videocited by 103
- 2017MovieGraphs: Towards Understanding Human-Centric Situations from Videoscited by 67
- 2020DramaQA: Character-Centered Video Story Understanding with Hierarchical QAcited by 45
- 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
- 2023AVscript: Accessible Video Editing with Audio-Visual Scriptscited by 19
- 2024ExpressEdit: Video Editing with Natural Language and Sketchingcited by 14
- 2025VideoDiff: Human-AI Video Co-Creation with Alternativescited by 8
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2025PPTArena: A Benchmark for PowerPoint Editingcited by 11
- 2026DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editingcited by 7
- 2026AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generationcited by 4
- 2026ELLMPEG: An Edge-based Agentic LLM Video Processing Toolcited by 1
- 2026BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generationcited by 1
- 2026Unified Agentic Video Editing Across Levels of Complexity and Creativitycited by 0
- 2026Unified Agentic Video Editing Across Levels of Complexity and Creativity11 Sep 2026
- 2026BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation26 May 2026
- 2026DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing6 Apr 2026
- 2026AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation30 Mar 2026
- 2026ELLMPEG: An Edge-based Agentic LLM Video Processing Tool17 Jan 2026
- 2025PPTArena: A Benchmark for PowerPoint Editing2 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2026VideoAgent: All-in-One Framework for Video Understanding and Editing
- 2025EditDuet: A Multi-Agent System for Video Non-Linear Editing
- 2026CutClaw: Agentic Hours-Long Video Editing via Music Synchronization
- 2026Camera Artist: A Multi-Agent Framework for Cinematic Language Storytelling Video Generation
- 2026StoryEngine: A State-Grounded Agentic Framework for Video Storytelling
- 2026The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation
- 2026WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation
- 2026ViMax: Agentic Video Generation