mlsanityData to 7 Oct 2026

Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion

arXiv 2407.01392 ↗First posted 1 Jul 2024cs.LG cs.CV cs.ROcited by 634

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

This paper presents Diffusion Forcing, a new training paradigm where a diffusion model is trained to denoise a set of tokens with independent per-token noise levels. We apply Diffusion Forcing to sequence generative modeling by training a causal next-token prediction model to generate one or several future tokens without fully diffusing past ones. Our approach is shown to combine the strengths of next-token prediction models, such as variable-length generation, with the strengths of full-sequence diffusion models, such as the ability to guide sampling to desirable trajectories. Our method offers a range of additional capabilities, such as (1) rolling-out sequences of continuous tokens, such as video, with lengths past the training horizon, where baselines diverge and (2) new sampling and guiding schemes that uniquely profit from Diffusion Forcing's variable-horizon and causal architecture, and which lead to marked performance gains in decision-making and planning tasks. In addition to its empirical success, our method is proven to optimize a variational lower bound on the likelihoods of all subsequences of tokens drawn from the true joint distribution. Project website: https://boyuan.space/diffusion-forcing

Key facts. Cited by 634 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 51 of its 70 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

51 of 70 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Tutorial on Diffusion Models for Imaging and Vision26 Mar
  2. 2023Diffusion Models as Masked Autoencoders6 Apr
  3. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  4. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  5. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  6. 2023On the Importance of Noise Scheduling for Diffusion Models26 Jan
  7. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  8. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
29 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  7. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  8. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  9. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  10. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  11. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  12. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  13. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  14. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  15. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  16. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  17. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  18. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  19. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  20. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  21. 2019High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processescited by 147
  22. 2020Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flowscited by 142
  23. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  24. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  25. 2024Rolling Diffusion Modelscited by 102
  26. 2022Temporally Consistent Transformers for Video Generationcited by 43
  27. 2021ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Modelscited by 42
  28. 2020Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulationcited by 37
  29. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behaviorcited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 244; 2026: 368.

634 papers.

Sort:

By area:

  1. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  2. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  3. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  4. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  5. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  6. 2026Causal World Modeling for Robot Controlcited by 289
  7. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  8. 2024Diffusion Policy Policy Optimizationcited by 255
  9. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  10. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  11. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  12. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
38 more
  1. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  2. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  3. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  4. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  5. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  6. 2025History-Guided Video Diffusioncited by 165
  7. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  8. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  9. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  10. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  11. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  12. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  13. 2026Advancing Open-source World Modelscited by 144
  14. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  15. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  16. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 124
  17. 2025Video World Models with Long-term Spatial Memorycited by 124
  18. 2025Training Agents Inside of Scalable World Modelscited by 122
  19. 2025Test-Time Training Done Rightcited by 121
  20. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  21. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  22. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  23. 2025Long Context Tuning for Video Generationcited by 110
  24. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  25. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  26. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  27. 2025Mixture of Contexts for Long Video Generationcited by 90
  28. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  29. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  30. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
  31. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  32. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 78
  33. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  34. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  35. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  36. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  37. 2025LongCat-Video Technical Reportcited by 73
  38. 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72

50 most cited of 634 All 634 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Token Time Continuous Diffusion for Language Modeling
  2. 2026Rethinking Token Prediction: Tree-Structured Diffusion Language Model
  3. 2025DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers
  4. 2026TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration
  5. 2025Unifying Autoregressive and Diffusion-Based Sequence Generation
  6. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
  7. 2026Keep The Essentials: Efficient Reference Conditioned Generation via Token Dropping
  8. 2026ALoDLM: Adaptively Looped Diffusion Language Models