# Pack and Force Your Memory: Long-form and Consistent Video Generation

arXiv: https://arxiv.org/abs/2510.01784 · HTML: https://mlsanity.com/paper/2510.01784/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 47 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Builds on (most cited first, 30 of 44)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://mlsanity.com/paper/2412.06464/) (2024)
- [Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation](https://mlsanity.com/paper/2312.13139/) (2023)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation](https://mlsanity.com/paper/2410.06158/) (2024)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025)
- [Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity](https://mlsanity.com/paper/2502.01776/) (2025)
- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200k](https://mlsanity.com/paper/2503.09642/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions](https://mlsanity.com/paper/2407.06358/) (2024)

## Built on by (most cited first, 29 of 29)

- [Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation](https://mlsanity.com/paper/2602.02214/) (2026)
- [Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2605.15141/) (2026)
- [Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models](https://mlsanity.com/paper/2603.25716/) (2026)
- [MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines](https://mlsanity.com/paper/2603.06679/) (2026)
- [AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories](https://mlsanity.com/paper/2602.14941/) (2026)
- [HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising](https://mlsanity.com/paper/2603.08703/) (2026)
- [Geometry-Aware Implicit Memory for Video World Models](https://mlsanity.com/paper/2606.02436/) (2026)
- [Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention](https://mlsanity.com/paper/2602.04789/) (2026)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation](https://mlsanity.com/paper/2605.30083/) (2026)
- [RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO](https://mlsanity.com/paper/2605.15190/) (2026)
- [Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory](https://mlsanity.com/paper/2605.18733/) (2026)
- [A Mechanistic View on Video Generation as World Models: State and Dynamics](https://mlsanity.com/paper/2601.17067/) (2026)
- [Echo-Memory: A Controlled Study of Memory in Action World Models](https://mlsanity.com/paper/2606.09803/) (2026)
- [Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion](https://mlsanity.com/paper/2608.26794/) (2026)
- [CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling](https://mlsanity.com/paper/2607.26529/) (2026)
- [EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation](https://mlsanity.com/paper/2605.23610/) (2026)
- [Surprise Forcing: What to Remember, When to Skip in Long Video Generation](https://mlsanity.com/paper/2607.18436/) (2026)
- [AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis](https://mlsanity.com/paper/2603.08021/) (2026)
- [Towards Memory-Efficient Autoregressive Video Generation via Instance-Specific Parametric Absorption](https://mlsanity.com/paper/2607.00712/) (2026)
- [Generative Augmented Reality: Paradigms, Technologies, and Future Applications](https://mlsanity.com/paper/2511.16783/) (2025)
- [WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory](https://mlsanity.com/paper/2609.24984/) (2026)
- [EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation](https://mlsanity.com/paper/2602.13669/) (2026)
- [Encore: Infinite Audio-Video Generation with Adaptive Signal Routing](https://mlsanity.com/paper/2609.04249/) (2026)
- [ViRDM: Taming Representation Distribution Matching for Few-Step Causal Video Generation](https://mlsanity.com/paper/2609.28923/) (2026)
- [DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency](https://mlsanity.com/paper/2609.39096/) (2026)
- [MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generation](https://mlsanity.com/paper/2610.02153/) (2026)
- [Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory](https://mlsanity.com/paper/2610.02521/) (2026)
- [GAD: Geometry Attention Distillation for Consistent Video Generation](https://mlsanity.com/paper/2610.18779/) (2026)

## Abstract (arXiv)

Long-form video generation presents a dual challenge: models must capture long-range dependencies while preventing the error accumulation inherent in autoregressive decoding. To address these challenges, we make two contributions. First, for dynamic context modeling, we propose MemoryPack, a learnable context-retrieval mechanism that leverages both textual and image information as global guidance to jointly model short- and long-term dependencies, achieving minute-level temporal consistency. This design scales gracefully with video length, preserves computational efficiency, and maintains linear complexity. Second, to mitigate error accumulation, we introduce Direct Forcing, an efficient single-step approximating strategy that improves training-inference alignment and thereby curtails error propagation during inference. Together, MemoryPack and Direct Forcing substantially enhance the context consistency and reliability of long-form video generation, advancing the practical usability of autoregressive video models.