# Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation

arXiv: https://arxiv.org/abs/2607.06631 · HTML: https://mlsanity.com/paper/2607.06631/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 27 of 27)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://mlsanity.com/paper/2411.19108/) (2024)
- [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025)
- [Diffusion Adversarial Post-Training for One-Step Video Generation](https://mlsanity.com/paper/2501.08316/) (2025)
- [Fast Video Generation with Sliding Tile Attention](https://mlsanity.com/paper/2502.04507/) (2025)
- [VSA: Faster Video Diffusion with Trainable Sparse Attention](https://mlsanity.com/paper/2505.13389/) (2025)
- [Faster gaze prediction with dense networks and Fisher pruning](https://mlsanity.com/paper/1801.05787/) (2018)
- [Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency](https://mlsanity.com/paper/2510.08431/) (2025)
- [Dynamic Diffusion Transformer](https://mlsanity.com/paper/2410.03456/) (2024)
- [SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention](https://mlsanity.com/paper/2509.24006/) (2025)
- [Pruning Convolutional Neural Networks for Resource Efficient Inference](https://mlsanity.com/paper/1611.06440/) (2016)
- [Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield](https://mlsanity.com/paper/2511.22677/) (2025)
- [TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times](https://mlsanity.com/paper/2512.16093/) (2025)
- [Transition Matching Distillation for Fast Video Generation](https://mlsanity.com/paper/2601.09881/) (2026)
- [Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals](https://mlsanity.com/paper/2510.27684/) (2025)
- [QVGen: Pushing the Limit of Quantized Video Generative Models](https://mlsanity.com/paper/2505.11497/) (2025)
- [DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder](https://mlsanity.com/paper/2509.25182/) (2025)
- [LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generation](https://mlsanity.com/paper/2510.08318/) (2025)
- [Learning to Skip the Middle Layers of Transformers](https://mlsanity.com/paper/2506.21103/) (2025)
- [SwiftVideo: A Unified Framework for Few-Step Video Generation through Trajectory-Distribution Alignment](https://mlsanity.com/paper/2508.06082/) (2025)
- [Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds](https://mlsanity.com/paper/2507.13343/) (2025)

## Abstract (arXiv)

Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.