mlsanityData to 7 Oct 2026

Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing

arXiv 2411.16375 ↗First posted 25 Nov 2024cs.CVcited by 70

On the map: Diffusion models → Video generation

Abstract (arXiv)

With the advance of diffusion models, today's video generation has achieved impressive quality. To extend the generation length and facilitate real-world applications, a majority of video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent clips conditioned on the last frame(s) of the previous clip. However, existing autoregressive VDMs are highly inefficient and redundant: The model must re-compute all the conditional frames that are overlapped between adjacent clips. This issue is exacerbated when the conditional frames are extended autoregressively to provide the model with long-term context. In such cases, the computational demands increase significantly (i.e., with a quadratic complexity w.r.t. the autoregression step). In this paper, we propose Ca2-VDM, an efficient autoregressive VDM with Causal generation and Cache sharing. For causal generation, it introduces unidirectional feature computation, which ensures that the cache of conditional frames can be precomputed in previous autoregression steps and reused in every subsequent step, eliminating redundant computations. For cache sharing, it shares the cache across all denoising steps to avoid the huge cache storage cost. Extensive experiments demonstrated that our Ca2-VDM achieves state-of-the-art quantitative and qualitative video generation results and significantly improves the generation speed. Code is available: https://github.com/Dawn-LX/CausalCache-VDM

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 53 of its 61 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

53 of 61 references.

Closest ancestors · 25

Same direction, newest first

  1. 2024Live2Diff: Live Stream Translation via Uni-directional Attention in Video Diffusion Models11 Jul
  2. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  3. 2024Latte: Latent Diffusion Transformer for Video Generation5 Jan
  4. 2023MEVG: Multi-event Video Generation with Text-to-Video Models7 Dec
  5. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Models30 Nov
  6. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  7. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models28 Nov
  8. 2023FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax27 Nov
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  10. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  11. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction31 Oct
  12. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling23 Oct
13 more
  1. 2023ModelScope Text-to-Video Technical Report12 Aug
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  3. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising29 May
  4. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modeling22 May
  5. 2023ControlVideo: Training-free Controllable Text-to-Video Generation22 May
  6. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  8. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  9. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation15 Mar
  10. 2023Video Probabilistic Diffusion Models in Projected Latent Space15 Feb
  11. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  12. 2022Flexible Diffusion Modeling of Long Videos23 May
  13. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation19 May

Other references · 28

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
16 more
  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  2. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  4. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  5. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  6. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  7. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  8. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  9. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  10. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  11. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  12. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  13. 2023AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidancecited by 66
  14. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 56
  15. 2023DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generationcited by 47
  16. 2023PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generatorscited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 52.

70 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2026World Action Models are Zero-shot Policiescited by 411
  3. 2025Video World Models with Long-term Spatial Memorycited by 124
  4. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  5. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  6. 2025Long-Context State-Space Video World Modelscited by 59
  7. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  8. 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 28
  9. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generationcited by 20
  10. 2025LoViC: Efficient Long Video Generation with Context Compressioncited by 19
  11. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 18
  12. 2026Streaming Autoregressive Video Generation via Diagonal Distillationcited by 15
38 more
  1. 2026LIVE: Long-horizon Interactive Video World Modelingcited by 14
  2. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 13
  3. 2026Past- and Future-Informed KV Cache Policy with Salience Estimation in Autoregressive Video Diffusioncited by 13
  4. 2026HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoisingcited by 13
  5. 2026Wonder: Video World Model Done Bettercited by 13
  6. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 12
  7. 2026Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Modelscited by 10
  8. 2026Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attentioncited by 9
  9. 2026Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generationcited by 9
  10. 2025CDP: Towards Robust Autoregressive Visuomotor Policy Learning via Causal Diffusioncited by 8
  11. 2026InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generationcited by 8
  12. 2025Real-Time Motion-Controllable Autoregressive Video Diffusioncited by 7
  13. 2025Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generationcited by 7
  14. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
  15. 2026TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignmentcited by 7
  16. 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
  17. 2025WorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusioncited by 6
  18. 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoisingcited by 6
  19. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  20. 2026Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Controlcited by 6
  21. 2026Head Forcing: Long Autoregressive Video Generation via Head Heterogeneitycited by 5
  22. 2026Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillationcited by 5
  23. 2026SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controlscited by 4
  24. 2026X-Cache: Cross-Chunk Block Caching for Few-Step Autoregressive World Models Inferencecited by 4
  25. 2026TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RLcited by 4
  26. 2026RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluationcited by 4
  27. 2026Vidu S1: A Real-Time Interactive Video Generation Modelcited by 4
  28. 2025Flow marching for a generative PDE foundation modelcited by 3
  29. 2025SpriteHand: Real-Time Versatile Hand-Object Interaction with Autoregressive Video Generationcited by 3
  30. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformerscited by 3
  31. 2026Repurposing 3D Generative Model for Autoregressive Layout Generationcited by 3
  32. 2026Exploring Data-Free LoRA Transferability for Video Diffusion Modelscited by 3
  33. 2026Stream-T1: Test-Time Scaling for Streaming Video Generationcited by 3
  34. 2026FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customizationcited by 3
  35. 2026ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Maskcited by 2
  36. 2026Latent Generative Solvers for Generalizable Long-Term Physics Simulationcited by 2
  37. 2026SoulX-LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memorycited by 2
  38. 2026LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generationcited by 2

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models
  2. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path
  3. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  4. 2024Progressive Autoregressive Video Diffusion Models
  5. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
  6. 2024S2DM: Sector-Shaped Diffusion Models for Video Generation
  7. 2026Flow caching for autoregressive video generation
  8. 2026Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation