mlsanityData to 7 Oct 2026

Denoising Diffusion Step-aware Models

arXiv 2310.03337 ↗First posted 5 Oct 2023cs.CVcited by 24

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Denoising Diffusion Probabilistic Models (DDPMs) have garnered popularity for data generation across various domains. However, a significant bottleneck is the necessity for whole-network computation during every step of the generative process, leading to high computational overheads. This paper presents a novel framework, Denoising Diffusion Step-aware Models (DDSM), to address this challenge. Unlike conventional approaches, DDSM employs a spectrum of neural networks whose sizes are adapted according to the importance of each generative step, as determined through evolutionary search. This step-wise network variation effectively circumvents redundant computational efforts, particularly in less critical steps, thereby enhancing the efficiency of the diffusion model. Furthermore, the step-aware design can be seamlessly integrated with other efficiency-geared diffusion models such as DDIMs and latent diffusion, thus broadening the scope of computational savings. Empirical evaluations demonstrate that DDSM achieves computational savings of 49% for CIFAR-10, 61% for CelebA-HQ, 59% for LSUN-bedroom, 71% for AFHQ, and 76% for ImageNet, all without compromising the generation quality.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 47 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

37 of 47 references.

Closest ancestors · 19

Same direction, newest first

  1. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  2. 2023AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration19 Sep
  3. 2023OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Models15 Jun
  4. 2023Structural Pruning for Diffusion Models18 May
  5. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  6. 2022Diffusion Probabilistic Model Made Slim27 Nov
  7. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  8. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2022Perception Prioritized Training of Diffusion Models1 Apr
  11. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
7 more
  1. 2021Score-based Generative Modeling in Latent Space10 Jun
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  4. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Implicit Models6 Oct
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  9. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  10. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  11. 2018Slimmable Neural Networkscited by 301
  12. 2019Universally Slimmable Networks and Improved Training Techniquescited by 219
6 more
  1. 2023DepGraph: Towards Any Structural Pruningcited by 202
  2. 2020pymoo: Multi-objective Optimization in Pythoncited by 196
  3. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  4. 2020Solving Linear Inverse Problems Using the Prior Implicit in a Denoisercited by 78
  5. 2020Powering One-shot Topological NAS with Stabilized Share-parameter Proxycited by 10
  6. 2021Fast and scalable neuroevolution deep learning architecture search for multivariate anomaly detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 9; 2026: 6.

24 papers.

Sort:

By area:

  1. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  2. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  3. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 138
  4. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 115
  5. 2024Attention-Driven Training-Free Efficiency Enhancement of Diffusion Modelscited by 37
  6. 2025MagCache: Fast Video Generation with Magnitude-Aware Cachecited by 36
  7. 2023Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inferencecited by 33
  8. 2024Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysiscited by 19
  9. 2024Structured Diffusion Models with Mixture of Gaussians as Prior Distributioncited by 8
  10. 2024Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Bettercited by 6
  11. 2025Cost-Aware Routing for Efficient Text-To-Image Generationcited by 4
  12. 2025Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearningcited by 2
12 more
  1. 2024Flexiffusion: Segment-wise Neural Architecture Search for Flexible Denoising Schedulecited by 1
  2. 2025Flexiffusion: Training-Free Segment-Wise Neural Architecture Search for Efficient Diffusion Modelscited by 1
  3. 2025BADiff: Bandwidth Adaptive Diffusion Modelcited by 1
  4. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Modelscited by 1
  5. 2025Fake it till You Make it: Reward Modeling as Discriminative Predictioncited by 0
  6. 2025Energy-Efficient Federated Learning for Edge Real-Time Vision via Joint Data, Computation, and Communication Designcited by 0
  7. 2025FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generationcited by 0
  8. 2026TMPDiff: Temporal Mixed-Precision for Diffusion Modelscited by 0
  9. 2026Magnitude-Direction Decoupling for Fast Video Generation with Flow Matching Modelscited by 0
  10. 2026SlimDiffuSE: Towards Efficient Diffusion-Based Speech Enhancement using Slimmable Networkscited by 0
  11. 2026Temporal State Transport in Video Generation: Diagnosing and Correcting Spectral Imbalancecited by 0
  12. 2026Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  2. 2024An overview of diffusion models for generative artificial intelligence
  3. 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis
  4. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models
  5. 2023The Missing U for Efficient Diffusion Models
  6. 2022Accelerating Diffusion Models via Early Stop of the Diffusion Process
  7. 2021Bilateral Denoising Diffusion Models
  8. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment