CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workflows remain largely underexplored. To bridge this gap, we introduce Cutverse, a benchmark designed to systematically evaluate autonomous GUI agents in realistic media post-production environments. We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop), covering 186 complex, long-horizon tasks grounded in authentic editing workflows, involving dense multimodal interfaces and tightly coupled interaction sequences. To support scalable evaluation, we develop a lightweight parser that transforms raw screen recordings and low-level interaction logs into structured, compositional GUI action trajectories with precise grounding. Extensive evaluations reveal that existing agents achieve only 36.0% task success on realistic media editing tasks, underscoring the challenges posed by complex, long-horizon media post-production workflows in our benchmark.While current models demonstrate promising spatial grounding, multimodal alignment, and coordinated action execution, they remain limited in long-horizon reliability and domain-specific planning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 51 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
41 of 51 references.
Closest ancestors · 14
Same direction, newest first
- 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data18 Sep
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
- 2025macOSWorld: A Multilingual Interactive Benchmark for GUI Agents4 Jun
- 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents17 Apr
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2025WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point12 Feb
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024GUI Agents: A Survey18 Dec
- 2024The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use15 Nov
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
- 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web27 Feb
2 more
Other references · 27
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025Qwen3 Technical Reportcited by 8,820
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
- 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 130
15 more
- 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
- 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 68
- 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methodscited by 54
- 2024LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editingcited by 47
- 2024DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusioncited by 47
- 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 37
- 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 36
- 2024VideoGUI: A Benchmark for GUI Automation from Instructional Videoscited by 35
- 2024ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systemscited by 30
- 2025FilMaster: Bridging Cinematic Principles and Generative AI for Automated Film Generationcited by 28
- 2025IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessmentcited by 28
- 2025MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generationcited by 23
- 2025ShotBench: Expert-Level Cinematic Understanding in Vision-Language Modelscited by 22
- 2025AniMaker: Multi-Agent Animated Storytelling with MCTS-Driven Clip Generationcited by 19
- 2025CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generationcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training
- 2026DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration
- 2026Covering Human Action Space for Computer Use: Data Synthesis and Benchmark
- 2026VideoAgent: All-in-One Framework for Video Understanding and Editing
- 2026Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents
- 2026Benchmarking and Improving GUI Agents in High-Dynamic Environments
- 2026Benchmarking Living-Screen-Native GUI Agents on Short-Video Platforms
- 2026Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields