mlsanityData to 7 Oct 2026

SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers

arXiv 2401.08740 ↗First posted 16 Jan 2024cs.CV cs.LGcited by 707

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present Scalable Interpolant Transformers (SiT), a family of generative models built on the backbone of Diffusion Transformers (DiT). The interpolant framework, which allows for connecting two distributions in a more flexible way than standard diffusion models, makes possible a modular study of various design choices impacting generative models built on dynamical transport: learning in discrete or continuous time, the objective function, the interpolant that connects the distributions, and deterministic or stochastic sampling. By carefully introducing the above ingredients, SiT surpasses DiT uniformly across model sizes on the conditional ImageNet 256x256 and 512x512 benchmark using the exact same model structure, number of parameters, and GFLOPs. By exploring various diffusion coefficients, which can be tuned separately from learning, SiT achieves an FID-50K score of 2.06 and 2.62, respectively.

Key facts. Cited by 707 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 64 of its 73 references that are in the map. First posted on arXiv 16 Jan 2024.

Builds on

64 of 73 references.

Closest ancestors · 33

Same direction, newest first

  1. 2023Non-Denoising Forward-Time Diffusions22 Dec
  2. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  3. 2023Error Bounds for Flow Matching Methods26 May
  4. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs6 May
  5. 2023Diffusion Schr"odinger Bridge Matching29 Mar
  6. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  7. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  8. 2023Consistency Models2 Mar
  9. 2023Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusions14 Feb
  10. 2023Minimizing Trajectory Curvature of ODE-based Generative Models27 Jan
  11. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  12. 2023On the Importance of Noise Scheduling for Diffusion Models26 Jan
21 more
  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  3. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  5. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  6. 2022Classifier-Free Diffusion Guidance26 Jul
  7. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  8. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  12. 2021Variational Diffusion Models1 Jul
  13. 2021Score-based Generative Modeling in Latent Space10 Jun
  14. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  16. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  17. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  18. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  19. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  20. 2020Denoising Diffusion Implicit Models6 Oct
  21. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
19 more
  1. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  3. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  4. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  5. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  6. 2018Image Transformercited by 770
  7. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  8. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  9. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  10. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 267
  11. 2022On Neural Differential Equationscited by 264
  12. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  13. 2023Flow Matching in Latent Spacecited by 155
  14. 2023Stochastic interpolants with data-dependent couplingscited by 94
  15. 2022Matching Normalizing Flows and Probability Paths on Manifoldscited by 57
  16. 2022A Self-Attention Ansatz for Ab-initio Quantum Chemistrycited by 45
  17. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 37
  18. 2023Multimarginal generative modeling with stochastic interpolantscited by 17
  19. 2023Deep learning probability flows and entropy production rates in active mattercited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 103; 2025: 301; 2026: 303.

707 papers.

Sort:

By area:

  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2025Mean Flows for One-step Generative Modelingcited by 524
  3. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  4. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  5. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  6. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  7. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  8. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  9. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
  10. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  11. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  12. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
38 more
  1. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  2. 2025Seedream 3.0 Technical Reportcited by 146
  3. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  4. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  5. 2024Randomized Autoregressive Visual Generationcited by 125
  6. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  7. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  8. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  9. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  10. 2025Inductive Moment Matchingcited by 107
  11. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  12. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  13. 2025DDT: Decoupled Diffusion Transformercited by 100
  14. 2024Flow map matching with stochastic interpolants: A mathematical framework for consistency modelscited by 99
  15. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 96
  16. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 95
  17. 2024ZigMa: A DiT-style Zigzag Mamba Diffusion Modelcited by 84
  18. 2025Latent Diffusion Model without Variational Autoencodercited by 83
  19. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  20. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 81
  21. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  22. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  23. 2025Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matchingcited by 78
  24. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  25. 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materialscited by 75
  26. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  27. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 75
  28. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  29. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
  30. 2025Diffuse and Disperse: Image Generation with Representation Regularizationcited by 67
  31. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  32. 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
  33. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 64
  34. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  35. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
  36. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Modelscited by 61
  37. 2024Dynamic Diffusion Transformercited by 60
  38. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55

50 most cited of 707 All 707 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From Navigation to Refinement: Revealing the Two-Stage Nature of Flow-based Diffusion Models through Oracle Velocity
  2. 2026Just-in-Time: Training-Free Spatial Acceleration for Diffusion Transformers
  3. 2024Scalable Diffusion Models with State Space Backbone
  4. 2023Improving Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architectures
  5. 2024Improving Diffusion-Based Generative Models via Approximated Optimal Transport
  6. 2025Multitask Learning with Stochastic Interpolants
  7. 2023State of the Art on Diffusion Models for Visual Computing
  8. 2026Synergizing Transport-Based Generative Models and Latent Geometry for Stochastic Closure Modeling