mlsanityData to 7 Oct 2026

Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion

arXiv 2410.19324 ↗First posted 25 Oct 2024cs.CV cs.LG stat.MLcited by 74

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Latent diffusion models have become the popular choice for scaling up diffusion models for high resolution image synthesis. Compared to pixel-space models that are trained end-to-end, latent models are perceived to be more efficient and to produce higher image quality at high resolution. Here we challenge these notions, and show that pixel-space models can be very competitive to latent models both in quality and efficiency, achieving 1.5 FID on ImageNet512 and new SOTA results on ImageNet128, ImageNet256 and Kinetics600. We present a simple recipe for scaling end-to-end pixel-space diffusion models to high resolutions. 1: Use the sigmoid loss-weighting (Kingma & Gao, 2023) with our prescribed hyper-parameters. 2: Use our simplified memory-efficient architecture with fewer skip-connections. 3: Scale the model to favor processing the image at a high resolution with fewer parameters, rather than using more parameters at a lower resolution. Combining these with guidance intervals, we obtain a family of pixel-space diffusion models we call Simpler Diffusion (SiD2).

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 47 of its 53 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

47 of 53 references.

Closest ancestors · 29

Same direction, newest first

  1. 2024MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning20 Aug
  2. 2024DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents3 Jul
  3. 2024Multistep Distillation of Diffusion Models via Moment Matching6 Jun
  4. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  5. 2024Improved Distribution Matching Distillation for Fast Image Synthesis23 May
  6. 2024PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher23 May
  7. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  8. 2024The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling23 Feb
  9. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  10. 2023DiffiT: Diffusion Vision Transformers for Image Generation4 Dec
  11. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  12. 2023Matryoshka Diffusion Models23 Oct
17 more
  1. 2023ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection20 Oct
  2. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion1 Oct
  3. 2023Relay Diffusion: Unifying diffusion process across resolutions for image synthesis4 Sep
  4. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  5. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  6. 2023On the Importance of Noise Scheduling for Diffusion Models26 Jan
  7. 2022Scalable Diffusion Models with Transformers19 Dec
  8. 2022f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation10 Oct
  9. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  13. 2021Variational Diffusion Models1 Jul
  14. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  15. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  16. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  17. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  10. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  11. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  12. 2022MAGVIT: Masked Generative Video Transformercited by 341
6 more
  1. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  2. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  3. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  4. 2024Rolling Diffusion Modelscited by 102
  5. 2024CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusioncited by 72
  6. 2023SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editingcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 36; 2026: 38.

74 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  3. 2025History-Guided Video Diffusioncited by 165
  4. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  5. 2026Generative Modeling via Driftingcited by 98
  6. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 91
  7. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 75
  8. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  9. 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 49
  10. 2025Fractal Generative Modelscited by 44
  11. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 42
  12. 2025Calibrated Multi-Preference Optimization for Aligning Diffusion Modelscited by 40
38 more
  1. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  2. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 33
  3. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 23
  4. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  5. 2026Unified Latents (UL): How to train your latentscited by 19
  6. 2025FARMER: Flow AutoRegressive Transformer over Pixelscited by 17
  7. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 16
  8. 2025Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scalescited by 13
  9. 2025Guiding a Diffusion Transformer with the Internal Dynamics of Itselfcited by 12
  10. 2026Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoderscited by 12
  11. 2025Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impactcited by 11
  12. 2025D-AR: Diffusion via Autoregressive Modelscited by 9
  13. 2025FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systemscited by 8
  14. 2026Autoregressive Image Generation with Masked Bit Modelingcited by 8
  15. 2026Image Generation with a Sphere Encodercited by 8
  16. 2025SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenizationcited by 7
  17. 2026Revisiting Diffusion Model Predictions Through Dimensionalitycited by 7
  18. 2026L2P: Unlocking Latent Potential for Pixel Generationcited by 7
  19. 2025Synthesis of discrete-continuous quantum circuits with multimodal diffusion modelscited by 6
  20. 2025Stochastic Interpolants via Conditional Dependent Couplingcited by 6
  21. 2025SimFlow: Simplified and End-to-End Training of Latent Normalizing Flowscited by 6
  22. 2026Laminating Representation Autoencoders for Efficient Diffusioncited by 6
  23. 2026End-to-End Training for Unified Tokenization and Latent Denoisingcited by 6
  24. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Valuecited by 5
  25. 2026GPIC: A Giant Permissive Image Corpus for Visual Generationcited by 5
  26. 2026Asymmetric Flow Modelscited by 4
  27. 2026Representation Forcing for Bottleneck-Free Unified Multimodal Modelscited by 4
  28. 2025Guidance Free Image Editing via Explicit Conditioningcited by 3
  29. 2025Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generationcited by 3
  30. 2025VUGEN: Visual Understanding priors for GENerationcited by 3
  31. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generationcited by 3
  32. 2026Normalizing Flows with Iterative Denoisingcited by 3
  33. 2026Forward-Learned Discrete Diffusion: Learning how to noise to denoise fastercited by 3
  34. 2025Latent Stochastic Interpolantscited by 2
  35. 2025Improving Multislice Electron Ptychography with a Generative Priorcited by 2
  36. 2025PAINT: Parallel-in-time Neural Twins for Dynamical System Reconstructioncited by 2
  37. 2026WIND: Weather Inverse Diffusion for Zero-Shot Atmospheric Modelingcited by 2
  38. 2026Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planningcited by 2

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Pixel-Space Post-Training of Latent Diffusion Models
  2. 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
  3. 2026Pixel-Space Diffusion Transformers
  4. 2025LowDiff: Efficient Diffusion Sampling with Low-Resolution Condition
  5. 2026A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples
  6. 2022Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance
  7. 2023Diffusion Models Without Attention
  8. 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models