# Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion

arXiv: https://arxiv.org/abs/2607.28058 · HTML: https://mlsanity.com/paper/2607.28058/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 19 of its 21 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 19 of 19)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive](https://mlsanity.com/paper/2402.13228/) (2024)
- [Unified Reward Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2503.05236/) (2025)
- [WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation](https://mlsanity.com/paper/2503.08153/) (2025)
- [MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models](https://mlsanity.com/paper/2501.02955/) (2025)
- [LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment](https://mlsanity.com/paper/2412.04814/) (2024)
- [OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimization](https://mlsanity.com/paper/2412.15159/) (2024)
- [DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2506.03517/) (2025)
- [Seeing Before Reasoning: A Unified Framework for Generalizable and Explainable Fake Image Detection](https://mlsanity.com/paper/2509.25502/) (2025)
- [OmniAID: Decoupling Semantics and Artifacts for Universal AI-Generated Image Detection in the Wild](https://mlsanity.com/paper/2511.08423/) (2025)
- [VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Rate](https://mlsanity.com/paper/2504.12259/) (2025)
- [Discriminator-Free Direct Preference Optimization for Video Diffusion](https://mlsanity.com/paper/2504.08542/) (2025)
- [RealDPO: Real or Not Real, that is the Preference](https://mlsanity.com/paper/2510.14955/) (2025)
- [DLFR-VAE: Dynamic Latent Frame Rate VAE for Video Generation](https://mlsanity.com/paper/2502.11897/) (2025)
- [Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2601.04068/) (2026)

## Abstract (arXiv)

Recent advances in preference alignment for diffusion-based video generation, particularly via Direct Preference Optimization (DPO), have significantly improved visual quality. However, temporally sparse artifacts such as motion collapse, object flickering, and color oversaturation remain a major barrier to perceptual realism. Existing methods struggle with these issues due to two key limitations: (1) the preference attribution bottleneck, where offline human annotations are costly and fail to accurately capture learning dynamics, while online reward signals are rollout-aware but often unstable and biased; and (2) temporal credit misallocation, where uniformly applied supervision cannot effectively target the brief segments in which artifacts occur. To address these challenges, we propose concentrated Implicit Preference Optimization (cIPO), a post-training framework for video diffusion models. cIPO derives implicit preference signals directly from the denoising process: given a real video, the model adds forward noise and reconstructs it via iterative denoising, treating the original as the preferred sample and the reconstruction as the dispreferred one. This formulation captures inference-time errors without requiring human annotations or external reward models. Moreover, frame-level discrepancies between original and reconstructed videos reveal when failures occur. cIPO leverages this by computing temporal reconstruction errors and concentrating optimization on high-error segments, enabling more precise correction of failure-prone regions. Extensive experiments demonstrate that cIPO consistently enhances video authenticity and temporal coherence across multiple datasets, highlighting the effectiveness and efficiency of implicit preference with temporally concentrated optimization.