mlsanityData to 14 Oct 2026

Prompt-Driven Agentic Video Editing System: Autonomous Comprehension of Long-Form, Story-Driven Media

arXiv 2509.16811 ↗First posted 20 Sep 2025cs.AI cs.HCcited by 6

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Creators struggle to edit long-form, narrative-rich videos not because of UI complexity, but due to the cognitive demands of searching, storyboarding, and sequencing hours of footage. Existing transcript- or embedding-based methods fall short for creative workflows, as models struggle to track characters, infer motivations, and connect dispersed events. We present a prompt-driven, modular editing system that helps creators restructure multi-hour content through free-form prompts rather than timelines. At its core is a semantic indexing pipeline that builds a global narrative via temporal segmentation, guided memory compression, and cross-granularity fusion, producing interpretable traces of plot, dialogue, emotion, and context. Users receive cinematic edits while optionally refining transparent intermediate outputs. Evaluated on 400+ videos with expert ratings, QA, and preference studies, our system scales prompt-driven editing, preserves narrative coherence, and balances automation with creator control.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 21 of its 43 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

21 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing15 Feb

Other references · 20

  1. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  2. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  3. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  4. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  5. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  6. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  7. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  8. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  9. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  10. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
  11. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  12. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
8 more
  1. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 183
  2. 2019Text-based Editing of Talking-head Videocited by 103
  3. 2017MovieGraphs: Towards Understanding Human-Centric Situations from Videoscited by 67
  4. 2020DramaQA: Character-Centered Video Story Understanding with Hierarchical QAcited by 45
  5. 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
  6. 2023AVscript: Accessible Video Editing with Audio-Visual Scriptscited by 19
  7. 2024ExpressEdit: Video Editing with Natural Language and Sketchingcited by 14
  8. 2025VideoDiff: Human-AI Video Co-Creation with Alternativescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026VideoAgent: All-in-One Framework for Video Understanding and Editing
  2. 2025EditDuet: A Multi-Agent System for Video Non-Linear Editing
  3. 2026CutClaw: Agentic Hours-Long Video Editing via Music Synchronization
  4. 2026Camera Artist: A Multi-Agent Framework for Cinematic Language Storytelling Video Generation
  5. 2026StoryEngine: A State-Grounded Agentic Framework for Video Storytelling
  6. 2026The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation
  7. 2026WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation
  8. 2026ViMax: Agentic Video Generation