mlsanityData to 14 Oct 2026

Speedrunning ImageNet Diffusion

arXiv 2512.12386 ↗First posted 13 Dec 2025cs.CVcited by 3

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recent advances have significantly improved the training efficiency of diffusion transformers. However, these techniques have largely been studied in isolation, leaving unexplored the potential synergies from combining multiple approaches. We present SR-DiT (Speedrun Diffusion Transformer), a framework that systematically integrates token routing, architectural improvements, and training modifications on top of representation alignment. Our approach achieves FID 3.49 and KDD 0.319 on ImageNet-256 using only a 140M parameter model at 400K iterations without classifier-free guidance - comparable to results from 685M parameter models trained significantly longer. To our knowledge, this is a state-of the-art result at this model size. Through extensive ablation studies, we identify which technique combinations are most effective and document both synergies and incompatibilities. We release our framework as a computationally accessible baseline for future research.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 35 of its 38 references that are in the map. First posted on arXiv 13 Dec 2025.

Builds on

35 of 38 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers24 Oct
  2. 2025Diffusion Transformers with Representation Autoencoders13 Oct
  3. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think2 Jul
  4. 2025Diffuse and Disperse: Image Generation with Representation Regularization10 Jun
  5. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers14 Apr
  6. 2025SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models11 Mar
  7. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Training8 Jan
  8. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models2 Jan
  9. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  10. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers4 May
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  12. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
4 more
  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2016Improved Techniques for Training GANscited by 2,970
  8. 2020GLU Variants Improve Transformercited by 1,804
  9. 2019Root Mean Square Layer Normalizationcited by 1,480
  10. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  11. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  12. 2020Query-Key Normalization for Transformerscited by 280
7 more
  1. 2021Generating Images with Sparse Representationscited by 266
  2. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 205
  3. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 164
  4. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 133
  5. 2025Contrastive Flow Matchingcited by 29
  6. 2025Equilibrium Matching: Generative Modeling with Implicit Energy-Based Modelscited by 21
  7. 2024Deriving Activation Functions Using Integrationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026LiFT: Loop Flow Transformerscited by 1
  2. 2026Motif-Video 2B: Technical Reportcited by 0
  3. 2026Image Tokenizers Should Be Trained Between Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification
  2. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  3. 2025Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts
  4. 2026MPDiT: Multi-Patch Global-to-Local Transformer Architecture For Efficient Flow Matching and Diffusion Model
  5. 2024Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers
  6. 2025DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers
  7. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  8. 2025Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas