# EditDuet: A Multi-Agent System for Video Non-Linear Editing

arXiv: https://arxiv.org/abs/2509.10761 · HTML: https://mlsanity.com/paper/2509.10761/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 14 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 6 of 6)

- [LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing](https://mlsanity.com/paper/2402.10294/) (2024)
- [AVscript: Accessible Video Editing with Audio-Visual Scripts](https://mlsanity.com/paper/2302.14117/) (2023)
- [B-Script: Transcript-based B-roll Video Editing with Recommendations](https://mlsanity.com/paper/1902.11216/) (2019)
- [ExpressEdit: Video Editing with Natural Language and Sketching](https://mlsanity.com/paper/2403.17693/) (2024)
- [Transcript to Video: Efficient Clip Sequencing from Texts](https://mlsanity.com/paper/2107.11851/) (2021)
- [PodReels: Human-AI Co-Creation of Video Podcast Teasers](https://mlsanity.com/paper/2311.05867/) (2023)

## Built on by (most cited first, 16 of 16)

- [DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing](https://mlsanity.com/paper/2604.04875/) (2026)
- [CutClaw: Agentic Hours-Long Video Editing via Music Synchronization](https://mlsanity.com/paper/2603.29664/) (2026)
- [A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation](https://mlsanity.com/paper/2604.10456/) (2026)
- [Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning](https://mlsanity.com/paper/2602.01335/) (2026)
- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing](https://mlsanity.com/paper/2604.05076/) (2026)
- [BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation](https://mlsanity.com/paper/2605.27067/) (2026)
- [Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation](https://mlsanity.com/paper/2608.02694/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making](https://mlsanity.com/paper/2602.11924/) (2026)
- [Autoregressive Modeling of Film with Applications in Video Montage](https://mlsanity.com/paper/2607.14645/) (2026)
- [VlogReward: Learning Multi-Dimensional Evaluation for Vlog Editing](https://mlsanity.com/paper/2607.22632/) (2026)
- [MOONWALK: Mediating Operations with Intent-Evidence-Action Alignment Across Junior-Supervisor Review Workflows in Animation/VFX Pre-Production](https://mlsanity.com/paper/2609.10385/) (2026)
- [Unified Agentic Video Editing Across Levels of Complexity and Creativity](https://mlsanity.com/paper/2609.12769/) (2026)
- [Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut](https://mlsanity.com/paper/2609.35143/) (2026)
- [Memory-Guided B-Roll Generation from User Video Collections](https://mlsanity.com/paper/2610.01884/) (2026)

## Abstract (arXiv)

Automated tools for video editing and assembly have applications ranging from filmmaking and advertisement to content creation for social media. Previous video editing work has mainly focused on either retrieval or user interfaces, leaving actual editing to the user. In contrast, we propose to automate the core task of video editing, formulating it as sequential decision making process. Ours is a multi-agent approach. We design an Editor agent and a Critic agent. The Editor takes as input a collection of video clips together with natural language instructions and uses tools commonly found in video editing software to produce an edited sequence. On the other hand, the Critic gives natural language feedback to the editor based on the produced sequence or renders it if it is satisfactory. We introduce a learning-based approach for enabling effective communication across specialized agents to address the language-driven video editing task. Finally, we explore an LLM-as-a-judge metric for evaluating the quality of video editing system and compare it with general human preference. We evaluate our system's output video sequences qualitatively and quantitatively through a user study and find that our system vastly outperforms existing approaches in terms of coverage, time constraint satisfaction, and human preference.