Mode Seeking meets Mean Seeking for Fast Long Video Generation
On the map: Diffusion models → Video generation
Abstract (arXiv)
Scaling video generation from seconds to minutes faces a critical bottleneck: while short-video data is abundant and high-fidelity, coherent long-form data is scarce and limited to narrow domains. To address this, we propose a training paradigm where Mode Seeking meets Mean Seeking, decoupling local fidelity from long-term coherence based on a unified representation via a Decoupled Diffusion Transformer. Our approach utilizes a global Flow Matching head trained via supervised learning on long videos to capture narrative structure, while simultaneously employing a local Distribution Matching head that aligns sliding windows to a frozen short-video teacher via a mode-seeking reverse-KL divergence. This strategy enables the synthesis of minute-scale videos that learns long-range coherence and motions from limited long videos via supervised flow matching, while inheriting local realism by aligning every sliding-window segment of the student to a frozen short-video teacher, resulting in a few-step fast long video generator. Evaluations show that our method effectively closes the fidelity-horizon gap by jointly improving local sharpness, motion and long-range consistency. Project website: https://primecai.github.io/mmm/.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 114 of its 126 references that are in the map. First posted on arXiv 27 Feb 2026.
Builds on
114 of 126 references.
Closest ancestors · 41
Same direction, newest first
- 2026SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning13 Feb
- 2026SLA2: Sparse-Linear Attention with Learnable Routing and QAT13 Feb
- 2026Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention4 Feb
- 2025StoryMem: Multi-shot Long Video Storytelling with Memory22 Dec
- 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resampling17 Dec
- 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compression4 Dec
- 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout25 Nov
- 2025MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation21 Oct
- 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recycling10 Oct
- 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generation2 Oct
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
29 more
- 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
- 2025Bidirectional Sparse Attention for Faster Video Diffusion Training1 Sep
- 2025StreamDiT: Real-Time Streaming Text-to-Video Generation4 Jul
- 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Models30 Jun
- 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generation24 Jun
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Models10 Jun
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion9 Jun
- 2025Test-Time Training Done Right29 May
- 2025StateSpaceDiffuser: Bringing Long Context to Diffusion World Models28 May
- 2025Long-Context State-Space Video World Models26 May
- 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation24 May
- 2025Training-Free Efficient Video Generation via Dynamic Token Carving22 May
- 2025VSA: Faster Video Diffusion with Trainable Sparse Attention19 May
- 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr
- 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models17 Apr
- 2025Long Context Tuning for Video Generation13 Mar
- 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generation28 Feb
- 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers21 Feb
- 2025Fast Video Generation with Sliding Tile Attention6 Feb
- 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
- 2025Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models14 Jan
- 2024LTX-Video: Realtime Video Latent Diffusion30 Dec
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
- 2024Pyramidal Flow Matching for Efficient Video Generative Modeling8 Oct
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention29 Jul
- 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
- 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling23 Oct
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
Other references · 73
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Consistency Modelscited by 1,778
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
38 more
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2022Token Merging: Your ViT But Fastercited by 995
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
- 2023Adversarial Diffusion Distillationcited by 741
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2025Mean Flows for One-step Generative Modelingcited by 524
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
- 2024One Step Diffusion via Shortcut Modelscited by 373
- 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
- 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
- 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
- 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
- 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
- 2025History-Guided Video Diffusioncited by 165
- 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
- 2025HunyuanVideo 1.5 Technical Reportcited by 148
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
- 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 113
- 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 113
- 2025FlexTok: Resampling Images into 1D Token Sequences of Flexible Lengthcited by 105
- 2024Rolling Diffusion Modelscited by 102
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2025One-Minute Video Generation with Test-Time Trainingcited by 100
- 2025Mixture of Contexts for Long Video Generationcited by 90
- 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
- 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 81
- 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
- 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 23
- 2026OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generatorscited by 12
- 2026Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Modelscited by 10
- 2026CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narrativescited by 8
- 2026Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillationcited by 5
- 2026DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillationcited by 4
- 2026AdaState: Self-Evolving Anchors for Streaming Video Generationcited by 2
- 2026OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generationcited by 2
- 2026Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rolloutcited by 2
- 2026DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generationcited by 0
- 2026In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusioncited by 0
- 2026LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generationcited by 0
- 2026LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation29 Sep 2026
- 2026In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion26 Sep 2026
- 2026Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout8 Sep 2026
- 2026Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation13 Aug 2026
- 2026DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation10 Aug 2026
- 2026DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation29 Jul 2026
- 2026OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators9 Jul 2026
- 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models24 Jun 2026
- 2026OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation28 May 2026
- 2026AdaState: Self-Evolving Anchors for Streaming Video Generation28 May 2026
- 2026CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives12 May 2026
- 2026Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models10 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching
- 2025FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation
- 2026Towards Error-Free Long Video Generation
- 2025FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation
- 2026TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion
- 2026Diff-VF: Training-free High-quality Long Video Generation via Diffusion Model
- 2026A2RD: Agentic Autoregressive Diffusion for Long Video Consistency
- 2026AtlasVid: Efficient Ultra-High-Resolution Long Video Generation via Decoupled Global-Local Modeling