mlsanityData to 7 Oct 2026

FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification

arXiv 2410.10356 ↗First posted 14 Oct 2024cs.CVcited by 64

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion Transformers (DiT) have attracted significant attention in research. However, they suffer from a slow convergence rate. In this paper, we aim to accelerate DiT training without any architectural modification. We identify the following issues in the training process: firstly, certain training strategies do not consistently perform well across different data. Secondly, the effectiveness of supervision at specific timesteps is limited. In response, we propose the following contributions: (1) We introduce a new perspective for interpreting the failure of the strategies. Specifically, we slightly extend the definition of Signal-to-Noise Ratio (SNR) and suggest observing the Probability Density Function (PDF) of SNR to understand the essence of the data robustness of the strategy. (2) We conduct numerous experiments and report over one hundred experimental results to empirically summarize a unified accelerating strategy from the perspective of PDF. (3) We develop a new supervision method that further accelerates the training process of DiT. Based on them, we propose FasterDiT, an exceedingly simple and practicable design strategy. With few lines of code modifications, it achieves 2.30 FID on ImageNet 256 resolution at 1000k iterations, which is comparable to DiT (2.27 FID) but 7 times faster in training.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 40 of its 46 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

40 of 46 references.

Closest ancestors · 21

Same direction, newest first

  1. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers9 May
  2. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformer25 Mar
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  4. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  5. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models10 Jan
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  7. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  8. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  9. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  10. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  11. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  12. 2023On the Importance of Noise Scheduling for Diffusion Models26 Jan
9 more
  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2022On Distillation of Guided Diffusion Models6 Oct
  3. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2022Video Diffusion Modelscited by 2,280
7 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  2. 2017Generative Adversarial Networks: An Overviewcited by 588
  3. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  4. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  5. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  6. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 102
  7. 2024Condition-Aware Neural Network for Controlled Image Generationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 32; 2026: 30.

64 papers.

Sort:

By area:

  1. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  2. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  3. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  4. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  5. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  6. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
  7. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 35
  8. 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Trainingcited by 33
  9. 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generationcited by 27
  10. 2025LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimizationcited by 20
  11. 2025Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generationcited by 20
  12. 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Modelscited by 16
38 more
  1. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 16
  2. 2025Turbo-VAED: Fast and Stable Transfer of Video-VAEs to Mobile Devicescited by 15
  3. 2024Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraintscited by 11
  4. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Modelscited by 10
  5. 2026PixelGen: Improving Pixel Diffusion with Perceptual Supervisioncited by 10
  6. 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 10
  7. 2025FerretNet: Efficient Synthetic Image Detection via Local Pixel Dependenciescited by 9
  8. 2025Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restorationcited by 8
  9. 2026Improving Reconstruction of Representation Autoencodercited by 8
  10. 2025Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltascited by 7
  11. 2025Decoupling High and Low Frequencies for Faithful Image Generation with Fine Detailscited by 7
  12. 2026What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusioncited by 7
  13. 2025Improved Training Technique for Shortcut Modelscited by 6
  14. 2025DINO-Tok: Adapting DINO for Visual Tokenizerscited by 6
  15. 2026SceneTok: A Compressed, Diffusable Token Space for 3D Scenescited by 6
  16. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Valuecited by 5
  17. 2025Distribution Matching Variational AutoEncodercited by 5
  18. 2025REGLUE Your Latents with Global and Local Semantics for Entangled Diffusioncited by 5
  19. 2025Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignmentcited by 4
  20. 2025MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devicescited by 4
  21. 2026Boosting Latent Diffusion Models via Disentangled Representation Alignmentcited by 4
  22. 2025Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attributioncited by 3
  23. 2025Spatial Reasoners for Continuous Variables in Any Domaincited by 3
  24. 2026UniX: Unifying Autoregression and Diffusion for Chest X-Ray Understanding and Generationcited by 3
  25. 2025Visual Generation Tuningcited by 2
  26. 2025Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimizationcited by 2
  27. 2026Pixel-Perfect Visual Geometry Estimationcited by 2
  28. 2026Geometric Autoencoder for Diffusion Modelscited by 2
  29. 2026Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editingcited by 2
  30. 2026IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncodercited by 2
  31. 2025RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Modelcited by 1
  32. 2026Guiding Token-Sparse Diffusion Modelscited by 1
  33. 2026Vision Foundation Models as Generalist Tokenizers for Image Generationcited by 1
  34. 2026FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformerscited by 1
  35. 2026UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reductioncited by 1
  36. 2026Unveiling the Secret of AdaLN-Zero in Diffusion Transformercited by 1
  37. 2025Fast and Stable Diffusion Planning through Variational Adaptive Weightingcited by 0
  38. 2025Fitting Image Diffusion Models on Video Datasetscited by 0

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  2. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  3. 2025RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformer
  4. 2025Speedrunning ImageNet Diffusion
  5. 2024Scaling Laws For Diffusion Transformers
  6. 2026Shiva-DiT: Residual-Based Differentiable Top-k Selection for Efficient Diffusion Transformers
  7. 2026Post-Training Pruning for Diffusion Transformers
  8. 2024TinyFusion: Diffusion Transformers Learned Shallow