# Uncertainty DMD: Restoring Diversity in Few-Step Autoregressive Video Distillation

arXiv: https://arxiv.org/abs/2609.11265 · HTML: https://mlsanity.com/paper/2609.11265/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 45 of its 61 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [LongLive: Real-time Interactive Long Video Generation](https://mlsanity.com/paper/2509.22622/) (2025)
- [Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models](https://mlsanity.com/paper/2305.18455/) (2023)
- [Rolling Forcing: Autoregressive Long Video Diffusion in Real Time](https://mlsanity.com/paper/2509.25161/) (2025)
- [Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation](https://mlsanity.com/paper/2602.02214/) (2026)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2506.09350/) (2025)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [One-Step Diffusion Distillation through Score Implicit Matching](https://mlsanity.com/paper/2410.16794/) (2024)
- [Helios: Real Real-Time Long Video Generation Model](https://mlsanity.com/paper/2603.04379/) (2026)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step](https://mlsanity.com/paper/2410.14919/) (2024)
- [AI Flow at the Network Edge](https://mlsanity.com/paper/2411.12469/) (2024)
- [Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield](https://mlsanity.com/paper/2511.22677/) (2025)
- [One-step Diffusion Models with f-Divergence Distribution Matching](https://mlsanity.com/paper/2502.15681/) (2025)
- [Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis](https://mlsanity.com/paper/2507.18569/) (2025)
- [Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Views](https://mlsanity.com/paper/2312.04424/) (2023)
- [LoL: Longer than Longer, Scaling Video Generation to Hour](https://mlsanity.com/paper/2601.16914/) (2026)
- [Distilling Diversity and Control in Diffusion Models](https://mlsanity.com/paper/2503.10637/) (2025)
- [Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation](https://mlsanity.com/paper/2508.03334/) (2025)
- [Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis](https://mlsanity.com/paper/2602.03139/) (2026)

## Abstract (arXiv)

Few-step distillation improves the efficiency of autoregressive (AR) video generation, but often causes diversity collapse: under the same prompt, different noise samples tend to produce highly similar videos with weakened motion dynamics. We analyze this degradation in Distribution Matching Distillation (DMD)-distilled AR video generators and find that, in the autoregressive setting, it takes the form of a structured uncertainty collapse: the mode-seeking bias of DMD maps different noise samples to nearly identical first chunks, and the deterministic AR cache then propagates this collapsed state to all subsequent chunks, turning a local loss of stochasticity at the rollout root into a global suppression of temporal variation. Based on this analysis, we propose Uncertainty DMD, a simple uncertainty-injection framework that restores stochasticity at two key stages of AR generation: a timestep perturbation for the first chunk to increase first-chunk diversity, and a stochastic cache-writing mechanism for later chunks to preserve uncertainty in autoregressive conditioning. The method requires no architectural changes and introduces only lightweight perturbation operations. The same perturbation mechanisms are used during both training and inference. Experiments show that Uncertainty DMD consistently improves diversity and motion dynamics while maintaining comparable per-sample visual quality.