Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation
Not on the map. Why?
Abstract (arXiv)
Long-horizon video editing agents receive final-product feedback only after many interdependent decisions. Yet editing quality is subjective, admits multiple valid solutions, and is not meaningfully calibrated across heterogeneous requests, making a global scalar objective both ambiguous and temporally uninformative. Our key observation is that fixing the request, materials, and production constraints converts this subjective objective into an ordinal comparison among directly comparable alternatives. We introduce Group-Relative Preference Backpropagation (GRPB), which transforms same-task rankings into zero-sum advantages and redistributes them as bounded credit over semantic editing segments. A lagged allocator and guarded transmission prevent current judgments or unreliable estimates from directly shaping the same rollout group. We manually construct a project-disjoint, horizon-stratified suite of realistic editing tasks for training and controlled evaluation. Across matched baselines, credit interventions, external benchmarking, and blinded human evaluation, GRPB improves both editing behavior and rendered products. The resulting 9B Crayotter model surpasses several proprietary systems on AgenticVBench, supporting task-local preference reduction as a practical approach to learning from subjective, delayed outcomes. Code and all supporting materials are publicly available at https://github.com/idwts/Crayotter.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
24 of 32 references.
References · 24
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2025Process Reinforcement through Implicit Rewardscited by 478
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
- 2018RUDDER: Return Decomposition for Delayed Rewardscited by 162
- 2024Free Process Rewards without Process Labelscited by 162
12 more
- 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 69
- 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 68
- 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 51
- 2025EditDuet: A Multi-Agent System for Video Non-Linear Editingcited by 16
- 2024Hindsight PRIORs for Reward Learning from Human Preferencescited by 12
- 2025From Outcomes to Processes: Guiding PRM Learning from ORM for Inference-Time Alignmentcited by 9
- 2024A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Modelcited by 8
- 2026DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editingcited by 7
- 2026AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?cited by 7
- 2026CutClaw: Agentic Hours-Long Video Editing via Music Synchronizationcited by 6
- 2026A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilationcited by 6
- 2026Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.