mlsanityData to 14 Oct 2026

Mixture of Contexts for Long Video Generation

arXiv 2508.21058 ↗First posted 28 Aug 2025cs.GR cs.AI cs.CVcited by 95

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Long video generation is fundamentally a long context memory problem: models must retain and retrieve salient events across a long range without collapsing or drifting. However, scaling diffusion transformers to generate long-context videos is fundamentally limited by the quadratic cost of self-attention, which makes memory and computation intractable and difficult to optimize for long sequences. We recast long-context video generation as an internal information retrieval task and propose a simple, learnable sparse attention routing module, Mixture of Contexts (MoC), as an effective long-term memory retrieval engine. In MoC, each query dynamically selects a few informative chunks plus mandatory anchors (caption, local windows) to attend to, with causal routing that prevents loop closures. As we scale the data and gradually sparsify the routing, the model allocates compute to salient history, preserving identities, actions, and scenes over minutes of content. Efficiency follows as a byproduct of retrieval (near-linear scaling), which enables practical training and synthesis, and the emergence of memory and consistency at the scale of minutes.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 62 of its 66 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

62 of 66 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  11. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
38 more
  1. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  2. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  3. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  4. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  5. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  6. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  7. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  8. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  9. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  10. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  11. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  12. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  13. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  14. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 251
  15. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  16. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 217
  17. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  18. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 171
  19. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
  20. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
  21. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  22. 2024In-Context LoRA for Diffusion Transformerscited by 158
  23. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
  24. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learnerscited by 130
  25. 2025Test-Time Training Done Rightcited by 130
  26. 2025Fast Video Generation with Sliding Tile Attentioncited by 128
  27. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 114
  28. 2025Long Context Tuning for Video Generationcited by 113
  29. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 109
  30. 2024Rolling Diffusion Modelscited by 104
  31. 2025VSA: Faster Video Diffusion with Trainable Sparse Attentioncited by 104
  32. 2025One-Minute Video Generation with Test-Time Trainingcited by 101
  33. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
  34. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 76
  35. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 69
  36. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 69
  37. 2024VideoStudio: Generating Consistent-Content and Multi-Scene Videoscited by 64
  38. 2025Long-Context State-Space Video World Modelscited by 64

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 72.

95 papers.

Sort:

By area:

  1. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  2. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 73
  3. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  4. 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 54
  5. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
  6. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  7. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 45
  8. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 42
  9. 2025StoryMem: Multi-shot Long Video Storytelling with Memorycited by 40
  10. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 36
  11. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 35
  12. 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 29
38 more
  1. 2026ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytellingcited by 28
  2. 2026SLA2: Sparse-Linear Attention with Learnable Routing and QATcited by 22
  3. 2025Bidirectional Sparse Attention for Faster Video Diffusion Trainingcited by 19
  4. 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
  5. 2025MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generationcited by 18
  6. 2026SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuningcited by 18
  7. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 16
  8. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 16
  9. 2026Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusioncited by 16
  10. 2026Pathwise Test-Time Correction for Autoregressive Long Video Generationcited by 15
  11. 2026Addressable Memory for Video World Modelscited by 15
  12. 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless Worldcited by 14
  13. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 13
  14. 2026Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attentioncited by 12
  15. 2026Mode Seeking meets Mean Seeking for Fast Long Video Generationcited by 12
  16. 2026Speed3R: Sparse Feed-forward 3D Reconstruction Modelscited by 10
  17. 2026SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routingcited by 10
  18. 2026CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narrativescited by 10
  19. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 10
  20. 2025BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generationcited by 9
  21. 2025LongVie 2: Multimodal Controllable Ultra-Long Video World Modelcited by 9
  22. 2026Plenoptic Video Generationcited by 9
  23. 2026MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generationcited by 9
  24. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 9
  25. 2025Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulationcited by 8
  26. 2025HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streamingcited by 8
  27. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 8
  28. 2026CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generationcited by 7
  29. 2026ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creationcited by 7
  30. 2026Evolution of Video Generative Foundationscited by 7
  31. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  32. 2026FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusioncited by 7
  33. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  34. 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generationcited by 6
  35. 2026EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generationcited by 6
  36. 2026ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocationcited by 6
  37. 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 6
  38. 2025From Masks to Worlds: A Hitchhiker's Guide to World Modelscited by 5

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos
  2. 2025SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding
  3. 2026LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation
  4. 2025AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding
  5. 2026VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
  6. 2026LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation
  7. 2025LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
  8. 2026OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation