# Early-Bird Diffusion: Investigating and Leveraging Timestep-Aware Early-Bird Tickets in Diffusion Models for Efficient Training arXiv: https://arxiv.org/abs/2504.09606 · HTML: https://mlsanity.com/paper/2504.09606/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 32 of its 42 references that are in the map. - First posted on arXiv 13 Apr 2025. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 32) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014) - [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018) - [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015) - [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022) - [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015) - [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022) - [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021) - [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019) - [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019) - [Perception Prioritized Training of Diffusion Models](https://mlsanity.com/paper/2204.00227/) (2022) - [Efficient Diffusion Training via Min-SNR Weighting Strategy](https://mlsanity.com/paper/2303.09556/) (2023) - [Structural Pruning for Diffusion Models](https://mlsanity.com/paper/2305.10924/) (2023) - [Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks](https://mlsanity.com/paper/1909.11957/) (2019) - [RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths](https://mlsanity.com/paper/2305.18295/) (2023) - [Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2304.12526/) (2023) - [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022) - [EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets](https://mlsanity.com/paper/2101.00063/) (2020) - [LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models](https://mlsanity.com/paper/2404.11098/) (2024) - [Patched Denoising Diffusion Models For High-Resolution Image Synthesis](https://mlsanity.com/paper/2308.01316/) (2023) - [Multi-Architecture Multi-Expert Diffusion Models](https://mlsanity.com/paper/2306.04990/) (2023) - [Addressing Negative Transfer in Diffusion Models](https://mlsanity.com/paper/2306.00354/) (2023) - [A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training](https://mlsanity.com/paper/2405.17403/) (2024) - [Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets](https://mlsanity.com/paper/2103.00794/) (2021) ## Built on by (most cited first, 5 of 5) - [Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment](https://mlsanity.com/paper/2508.06160/) (2025) - [FasterVAR: Plug-and-Play Acceleration for Visual Autoregressive Models](https://mlsanity.com/paper/2512.16483/) (2025) - [HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editing](https://mlsanity.com/paper/2606.13898/) (2026) - [TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models](https://mlsanity.com/paper/2608.03057/) (2026) - [Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling](https://mlsanity.com/paper/2609.16450/) (2026) ## Abstract (arXiv) Training diffusion models (DMs) requires substantial computational resources due to multiple forward and backward passes across numerous timesteps, motivating research into efficient training techniques. In this paper, we propose EB-Diff-Train, a new efficient DM training approach that is orthogonal to other methods of accelerating DM training, by investigating and leveraging Early-Bird (EB) tickets -- sparse subnetworks that manifest early in the training process and maintain high generation quality. We first investigate the existence of traditional EB tickets in DMs, enabling competitive generation quality without fully training a dense model. Then, we delve into the concept of diffusion-dedicated EB tickets, drawing on insights from varying importance of different timestep regions. These tickets adapt their sparsity levels according to the importance of corresponding timestep regions, allowing for aggressive sparsity during non-critical regions while conserving computational resources for crucial timestep regions. Building on this, we develop an efficient DM training technique that derives timestep-aware EB tickets, trains them in parallel, and combines them during inference for image generation. Extensive experiments validate the existence of both traditional and timestep-aware EB tickets, as well as the effectiveness of our proposed EB-Diff-Train method. This approach can significantly reduce training time both spatially and temporally -- achieving 2.9× to 5.8× speedups over training unpruned dense models, and up to 10.3× faster training compared to standard train-prune-finetune pipelines -- without compromising generative quality. Our code is available at https://github.com/GATECH-EIC/Early-Bird-Diffusion.