mlsanityData to 7 Oct 2026

TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation

arXiv 2303.04248 ↗First posted 7 Mar 2023cs.LG cs.CVcited by 115

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.

Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 41 of its 50 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

41 of 50 references.

Closest ancestors · 16

Same direction, newest first

  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  3. 2022f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation10 Oct
  4. 2022On Distillation of Guided Diffusion Models6 Oct
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  9. 2021Score-based Generative Modeling in Latent Space10 Jun
  10. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  11. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
4 more
  1. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Implicit Models6 Oct
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2018Neural Ordinary Differential Equationscited by 3,733
  9. 2015Continuous control with deep reinforcement learningcited by 3,279
  10. 2016Density estimation using Real NVPcited by 2,029
  11. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
13 more
  1. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  4. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  5. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  6. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  7. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  8. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  9. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  10. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  11. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  12. 2021NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsamplingcited by 50
  13. 2022NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Ratescited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 44; 2025: 44; 2026: 6.

115 papers.

Sort:

By area:

  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  3. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  4. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 448
  5. 2023Improved Techniques for Training Consistency Modelscited by 430
  6. 2024One Step Diffusion via Shortcut Modelscited by 373
  7. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  8. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  9. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  10. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 205
  11. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  12. 2024Consistency Models Made Easycited by 140
38 more
  1. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  2. 2025Inductive Moment Matchingcited by 107
  3. 2024Improving the Training of Rectified Flowscited by 106
  4. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrappingcited by 96
  5. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 95
  6. 2023Fast ODE-based Sampling for Diffusion Models in Around 5 Stepscited by 83
  7. 2024Phased Consistency Modelscited by 83
  8. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  9. 2024Distilling Diffusion Models into Conditional GANscited by 78
  10. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 76
  11. 2023AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Accelerationcited by 70
  12. 2024Multistep Consistency Modelscited by 67
  13. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  14. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 60
  15. 2024EM Distillation for One-step Diffusion Modelscited by 60
  16. 2023Swallowing the Bitter Pill: Simplified Scalable Conformer Generationcited by 55
  17. 2024Adversarial Diffusion Compression for Real-World Image Super-Resolutioncited by 49
  18. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  19. 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
  20. 2025AlphaFlow: Understanding and Improving MeanFlow Modelscited by 40
  21. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  22. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  23. 2025FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generationcited by 32
  24. 2024Simple and Fast Distillation of Diffusion Modelscited by 31
  25. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  26. 2024SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flowcited by 30
  27. 2025Accelerated Diffusion Models via Speculative Samplingcited by 30
  28. 2024You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANscited by 25
  29. 2024Truncated Consistency Modelscited by 21
  30. 2025Dual-Expert Consistency Model for Efficient and High-Quality Video Generationcited by 21
  31. 2023CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulationcited by 20
  32. 2023Continual Learning of Diffusion Models with Generative Distillationcited by 19
  33. 2024Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysiscited by 19
  34. 2023Clockwork Diffusion: Efficient Generation With Model-Step Distillationcited by 18
  35. 2024Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminatorcited by 18
  36. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  37. 2023A Geometric Perspective on Diffusion Modelscited by 17
  38. 2023DiffEnc: Variational Diffusion with a Learned Encodercited by 16

50 most cited of 115 All 115 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models
  2. 2023Fast Inference in Denoising Diffusion Models via MMD Finetuning
  3. 2024Diffusion Bridge Implicit Models
  4. 2022Deep Equilibrium Approaches to Diffusion Models
  5. 2022Progressive Distillation for Fast Sampling of Diffusion Models
  6. 2023Simple diffusion: End-to-end diffusion for high resolution images
  7. 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation
  8. 2024Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization