# AgentMV: A State-Guided Multi-Agent Framework for Budget-Aware Music Video Generation

arXiv: https://arxiv.org/abs/2605.10723 · HTML: https://mlsanity.com/paper/2605.10723/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 25 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 21 of 21)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025)
- [StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation](https://mlsanity.com/paper/2405.01434/) (2024)
- [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024)
- [BlockDrop: Dynamic Inference Paths in Residual Networks](https://mlsanity.com/paper/1711.08393/) (2017)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling](https://mlsanity.com/paper/2310.15169/) (2023)
- [MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making](https://mlsanity.com/paper/2404.15155/) (2024)
- [Seedream 3.0 Technical Report](https://mlsanity.com/paper/2504.11346/) (2025)
- [Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising](https://mlsanity.com/paper/2305.18264/) (2023)
- [Adaptive Caching for Faster Video Generation with Diffusion Transformers](https://mlsanity.com/paper/2411.02397/) (2024)
- [SongFormer: Scaling Music Structure Analysis with Heterogeneous Supervision](https://mlsanity.com/paper/2510.02797/) (2025)
- [AutoMV: An Automatic Multi-Agent System for Music Video Generation](https://mlsanity.com/paper/2512.12196/) (2025)
- [Music2Video: Automatic Generation of Music Video with fusion of audio and text](https://mlsanity.com/paper/2201.03809/) (2022)

## Abstract (arXiv)

Generating a complete music video from a song requires more than synthesizing visually plausible clips for individual lyric prompts. A practical system must maintain long-range visual consistency, coordinate recurring motifs, synchronize edits with musical structure, and manage the cumulative cost of video generation. Existing approaches typically generate segments independently or adopt fixed generation strategies, limiting their ability to perform global planning over an entire song. We present AgentMV, a state-guided multi-agent framework for budget-aware music video generation. AgentMV decomposes the production process into specialized agents for music perception, script planning, visual asset provision, segment realization, and final assembly, coordinated through a Structured Persistent State that enables information exchange and state tracking throughout the generation process. To optimize generation resources, we formulate motif-aware segment realization as a group-level Multiple-Choice Knapsack Problem solved via dynamic programming, considering segment importance, generation quality, cost, and motif reuse under a global budget constraint. Experiments on song benchmarks demonstrate that AgentMV improves quality-cost trade-offs over existing MV generation frameworks, highlighting the potential of state-guided multi-agent coordination and budget-aware planning for long-form music video generation with improved quality and efficiency.