mlsanityData to 7 Oct 2026

Ditto: Accelerating Diffusion Model via Temporal Value Similarity

arXiv 2501.11211 ↗First posted 20 Jan 2025cs.AR cs.CV cs.LGcited by 18

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models achieve superior performance in image generation tasks. However, it incurs significant computation overheads due to its iterative structure. To address these overheads, we analyze this iterative structure and observe that adjacent time steps in diffusion models exhibit high value similarity, leading to narrower differences between consecutive time steps. We adapt these characteristics to a quantized diffusion model and reveal that the majority of these differences can be represented with reduced bit-width, and even zero. Based on our observations, we propose the Ditto algorithm, a difference processing algorithm that leverages temporal similarity with quantization to enhance the efficiency of diffusion models. By exploiting the narrower differences and the distributive property of layer operations, it performs full bit-width operations for the initial time step and processes subsequent steps with temporal differences. In addition, Ditto execution flow optimization is designed to mitigate the memory overhead of temporal difference processing, further boosting the efficiency of the Ditto algorithm. We also design the Ditto hardware, a specialized hardware accelerator, fully exploiting the dynamic characteristics of the proposed algorithm. As a result, the Ditto hardware achieves up to 1.5x speedup and 17.74% energy saving compared to other accelerators.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 55 of its 96 references that are in the map. First posted on arXiv 20 Jan 2025.

Builds on

55 of 96 references.

Closest ancestors · 19

Same direction, newest first

  1. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  2. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  3. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  4. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2022Post-training Quantization on Diffusion Models28 Nov
  7. 2022Diffusion Models in Vision: A Survey10 Sep
  8. 2022A Survey on Generative Diffusion Model6 Sep
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  10. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  11. 2022Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders19 Feb
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
7 more
  1. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  2. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  3. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  4. 2021Noise Estimation for Generative Diffusion Models6 Apr
  5. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  6. 2020Denoising Diffusion Implicit Models6 Oct
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 36

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  11. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  12. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
24 more
  1. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  2. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  3. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  4. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  5. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  6. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  7. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  8. 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 142
  9. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  10. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 95
  11. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 83
  12. 2020Convolutional Tensor-Train LSTM for Spatio-temporal Learningcited by 54
  13. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  14. 2018Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarkingcited by 35
  15. 2016Bit-pragmatic Deep Neural Network Computingcited by 30
  16. 2020Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Frameworkcited by 27
  17. 2021Dual-side Sparse Tensor Corecited by 27
  18. 2020EdgeDRNN: Recurrent Neural Network Accelerator for Edge Inferencecited by 19
  19. 2021S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Accelerationcited by 19
  20. 2022Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Modelscited by 16
  21. 2018E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAscited by 8
  22. 2023Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloadscited by 6
  23. 2023Generative AI Beyond LLMs: System Implications of Multi-Modal Generationcited by 5
  24. 2021MERCURY: Accelerating DNN Training By Exploiting Input Similaritycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 12.

18 papers.

Sort:

By area:

  1. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 124
  2. 2025SpeCa: Accelerating Diffusion Transformers with Speculative Feature Cachingcited by 27
  3. 2025Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformerscited by 23
  4. 2025Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Cachingcited by 17
  5. 2025FreqCa: Accelerating Diffusion Models via Frequency-Aware Cachingcited by 16
  6. 2025Let Features Decide Their Own Solvers: Hybrid Feature Caching for Diffusion Transformerscited by 8
  7. 2026TC-Pad'e: Trajectory-Consistent Pad'e Approximation for Diffusion Accelerationcited by 8
  8. 2026Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCachecited by 5
  9. 2026Adaptive Spectral Feature Forecasting for Diffusion Sampling Accelerationcited by 5
  10. 2026DisCa: Accelerating Video Diffusion Transformers with Distillation-Compatible Learnable Feature Cachingcited by 4
  11. 2026TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Accelerationcited by 3
  12. 2026ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Accelerationcited by 3
6 more
  1. 2026Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Efficient Diffusion Transformerscited by 2
  2. 2026LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Accelerationcited by 1
  3. 2026Dynamic Video Generation: Shaping Video Generation Across Time and Spacecited by 0
  4. 2026Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlationscited by 0
  5. 2026Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inferencecited by 0
  6. 2026LinCa: Accelerating Diffusion Models via Learnable Decomposed Feature Cachingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
  2. 2025DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference
  3. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  4. 2025DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers
  5. 2023AdaDiff: Accelerating Diffusion Models through Step-Wise Adaptive Computation
  6. 2026DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse
  7. 2023One-step Diffusion with Distribution Matching Distillation
  8. 2026Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness