# Context Forcing: Consistent Autoregressive Video Generation with Long Context

arXiv: https://arxiv.org/abs/2602.06028 · HTML: https://mlsanity.com/paper/2602.06028/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 49 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 30 of 46)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [Pyramidal Flow Matching for Efficient Video Generative Modeling](https://mlsanity.com/paper/2410.05954/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [LongLive: Real-time Interactive Long Video Generation](https://mlsanity.com/paper/2509.22622/) (2025)
- [Rolling Forcing: Autoregressive Long Video Diffusion in Real Time](https://mlsanity.com/paper/2509.25161/) (2025)
- [SkyReels-V2: Infinite-length Film Generative Model](https://mlsanity.com/paper/2504.13074/) (2025)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [Self-Forcing++: Towards Minute-Scale High-Quality Video Generation](https://mlsanity.com/paper/2510.02283/) (2025)
- [Autoregressive Video Generation without Vector Quantization](https://mlsanity.com/paper/2412.14169/) (2024)
- [WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling](https://mlsanity.com/paper/2512.14614/) (2025)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [FIFO-Diffusion: Generating Infinite Videos from Text without Training](https://mlsanity.com/paper/2405.11473/) (2024)

## Built on by (most cited first, 30 of 59)

- [LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation](https://mlsanity.com/paper/2605.18739/) (2026)
- [MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens](https://mlsanity.com/paper/2603.12513/) (2026)
- [Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation](https://mlsanity.com/paper/2603.21366/) (2026)
- [Self Gradient Forcing: Native Long Video Extrapolation](https://mlsanity.com/paper/2607.20368/) (2026)
- [LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation](https://mlsanity.com/paper/2606.02553/) (2026)
- [ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU](https://mlsanity.com/paper/2607.19191/) (2026)
- [Addressable Memory for Video World Models](https://mlsanity.com/paper/2608.07408/) (2026)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing](https://mlsanity.com/paper/2603.08982/) (2026)
- [Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation](https://mlsanity.com/paper/2606.04527/) (2026)
- [Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation](https://mlsanity.com/paper/2605.30083/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO](https://mlsanity.com/paper/2605.15190/) (2026)
- [Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory](https://mlsanity.com/paper/2605.18733/) (2026)
- [AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising](https://mlsanity.com/paper/2603.14331/) (2026)
- [Grounded Forcing: Bridging Time-Independent Semantics and Proximal Dynamics in Autoregressive Video Synthesis](https://mlsanity.com/paper/2604.06939/) (2026)
- [A2RD: Agentic Autoregressive Diffusion for Long Video Consistency](https://mlsanity.com/paper/2605.06924/) (2026)
- [Compression and Retrieval: Implicit Memory Retrieval for Video World Models](https://mlsanity.com/paper/2606.23105/) (2026)
- [AlayaWorld: Long-Horizon and Playable Video World Generation](https://mlsanity.com/paper/2607.06291/) (2026)
- [Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity](https://mlsanity.com/paper/2605.14487/) (2026)
- [DIM-WAM: World-Action Modeling with Diverse Historical Event Memory](https://mlsanity.com/paper/2606.27677/) (2026)
- [Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation](https://mlsanity.com/paper/2608.13391/) (2026)
- [AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model](https://mlsanity.com/paper/2604.19747/) (2026)
- [DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation](https://mlsanity.com/paper/2607.26811/) (2026)
- [LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation](https://mlsanity.com/paper/2608.28460/) (2026)
- [EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration](https://mlsanity.com/paper/2605.15042/) (2026)
- [RECAP-Forcing: Retaining Content Appearances for Long Video Generation](https://mlsanity.com/paper/2608.26671/) (2026)
- [Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generation](https://mlsanity.com/paper/2605.14382/) (2026)
- [Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion](https://mlsanity.com/paper/2605.18346/) (2026)
- [Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation](https://mlsanity.com/paper/2605.20476/) (2026)

## Abstract (arXiv)

Recent approaches to real-time long video generation typically employ streaming tuning strategies, attempting to train a long-context student using a short-context (memoryless) teacher. In these frameworks, the student performs long rollouts but receives supervision from a teacher limited to short 5-second windows. This structural discrepancy creates a critical student-teacher mismatch: the teacher's inability to access long-term history prevents it from guiding the student on global temporal dependencies, effectively capping the student's context length. To resolve this, we propose Context Forcing, a novel framework that trains a long-context student via a long-context teacher. By ensuring the teacher is aware of the full generation history, we eliminate the supervision mismatch, enabling the robust training of models capable of long-term consistency. To make this computationally feasible for extreme durations (e.g., 2 minutes), we introduce a context management system that transforms the linearly growing context into a Slow-Fast Memory architecture, significantly reducing visual redundancy. Extensive results demonstrate that our method enables effective context lengths exceeding 20 seconds -- 2 to 10 times longer than state-of-the-art methods like LongLive and Infinite-RoPE. By leveraging this extended context, Context Forcing preserves superior consistency across long durations, surpassing state-of-the-art baselines on various long video evaluation metrics.