mlsanityData to 7 Oct 2026

Long-Context Autoregressive Video Modeling with Next-Frame Prediction

arXiv 2503.19325 ↗First posted 25 Mar 2025cs.CVcited by 119

Not on the map. Why?

Cited by 47 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Long-context video modeling is essential for enabling generative models to function as world simulators, as they must maintain temporal coherence over extended time spans. However, most existing models are trained on short clips, limiting their ability to capture long-range dependencies, even with test-time extrapolation. While training directly on long videos is a natural solution, the rapid growth of vision tokens makes it computationally prohibitive. To support exploring efficient long-context video modeling, we first establish a strong autoregressive baseline called Frame AutoRegressive (FAR). FAR models temporal dependencies between continuous frames, converges faster than video diffusion transformers, and outperforms token-level autoregressive models. Based on this baseline, we observe context redundancy in video autoregression. Nearby frames are critical for maintaining temporal consistency, whereas distant frames primarily serve as context memory. To eliminate this redundancy, we propose the long short-term context modeling using asymmetric patchify kernels, which apply large kernels to distant frames to reduce redundant tokens, and standard kernels to local frames to preserve fine-grained detail. This significantly reduces the training cost of long videos. Our method achieves state-of-the-art results on both short and long video generation, providing an effective baseline for long-context autoregressive video modeling.

Key facts. Cited by 119 papers in the map (data through 7 Oct 2026). Builds on 61 of its 75 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

61 of 75 references.

References · 61

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  12. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
38 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  3. 2025s1: Simple test-time scalingcited by 1,311
  4. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  10. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  11. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  12. 2024Genie: Generative Interactive Environmentscited by 706
  13. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  14. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  15. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  16. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  17. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  18. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  19. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  20. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  21. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  22. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  23. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  24. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  25. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  26. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  27. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
  28. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  29. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  30. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  31. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
  32. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  33. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 137
  34. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  35. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  36. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  37. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  38. 2025Long Context Tuning for Video Generationcited by 110

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 56; 2026: 63.

119 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  4. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  5. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  6. 2025Video World Models with Long-term Spatial Memorycited by 124
  7. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  8. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  9. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  10. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  11. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 68
  12. 2025Long-Context State-Space Video World Modelscited by 59
38 more
  1. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 58
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 56
  3. 2025Captain Cinema: Towards Short Movie Generationcited by 48
  4. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 47
  5. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 40
  6. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  7. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  8. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  9. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 31
  10. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 31
  11. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 30
  12. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  13. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  14. 2026AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillationcited by 23
  15. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  16. 2025Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesiscited by 20
  17. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 20
  18. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generationcited by 20
  19. 2025LoViC: Efficient Long Video Generation with Context Compressioncited by 19
  20. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusioncited by 18
  21. 2025MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generationcited by 18
  22. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 18
  23. 2025Generative Pre-trained Autoregressive Diffusion Transformercited by 16
  24. 2025MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generationcited by 16
  25. 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusioncited by 15
  26. 2025DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencodercited by 15
  27. 2025Vidarc: Embodied Video Diffusion Model for Closed-loop Controlcited by 15
  28. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generationcited by 15
  29. 2026Efficient Autoregressive Video Diffusion with Dummy Headcited by 15
  30. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 15
  31. 2026Streaming Autoregressive Video Generation via Diagonal Distillationcited by 15
  32. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
  33. 2025WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perceptioncited by 14
  34. 2025Personalized Vision via Visual In-Context Learningcited by 14
  35. 2026LIVE: Long-horizon Interactive Video World Modelingcited by 14
  36. 2026Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusioncited by 14
  37. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 13
  38. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13

50 most cited of 119 All 119 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Recency Forcing: Bridging the Long-Horizon Gap in Autoregressive Video Generation
  2. 2026TokenTrim: Inference-Time Token Pruning for Autoregressive Long Video Generation
  3. 2026In-Context Forcing: Uncovering Context Effects in Autoregressive Video Diffusion
  4. 2026Self Gradient Forcing: Native Long Video Extrapolation
  5. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resampling
  6. 2026VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
  7. 2025Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations
  8. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generation