mlsanityData to 14 Oct 2026

Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers

arXiv 2401.11605 ↗First posted 21 Jan 2024cs.CV cs.AI cs.LGcited by 118

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present the Hourglass Diffusion Transformer (HDiT), an image generative model that exhibits linear scaling with pixel count, supporting training at high-resolution (e.g. 1024 × 1024) directly in pixel-space. Building on the Transformer architecture, which is known to scale to billions of parameters, it bridges the gap between the efficiency of convolutional U-Nets and the scalability of Transformers. HDiT trains successfully without typical high-resolution training techniques such as multiscale architectures, latent autoencoders or self-conditioning. We demonstrate that HDiT performs competitively with existing models on ImageNet 2562, and sets a new state-of-the-art for diffusion models on FFHQ-10242.

Key facts. Cited by 118 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 53 of its 69 references that are in the map. First posted on arXiv 21 Jan 2024.

Builds on

53 of 69 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023Boosting Latent Diffusion with Flow Matching12 Dec
  2. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  4. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  5. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  6. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  7. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale12 Mar
  8. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  9. 2022Scalable Diffusion Models with Transformers19 Dec
  10. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  11. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model16 Aug
  12. 2022Classifier-Free Diffusion Guidance26 Jul
8 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2016Layer Normalizationcited by 5,669
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  11. 2020GLU Variants Improve Transformercited by 1,804
  12. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
21 more
  1. 2019Root Mean Square Layer Normalizationcited by 1,480
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  5. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  7. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  8. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  9. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
  10. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  11. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  12. 2019Axial Attention in Multidimensional Transformerscited by 358
  13. 2020Query-Key Normalization for Transformerscited by 280
  14. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  15. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  16. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
  17. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  18. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  19. 2021Hierarchical Transformers Are More Efficient Language Modelscited by 102
  20. 2021Improved Transformer for High-Resolution GANscited by 59
  21. 2022Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learningcited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 35; 2025: 38; 2026: 45.

118 papers.

Sort:

By area:

  1. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  2. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 299
  3. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
  4. 2024Consistency Models Made Easycited by 144
  5. 2024ZigMa: A DiT-style Zigzag Mamba Diffusion Modelcited by 84
  6. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 54
  7. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50
  8. 2024TEXGen: a Generative Diffusion Model for Mesh Texturescited by 46
  9. 2024Improved Noise Schedule for Diffusion Trainingcited by 39
  10. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 36
  11. 2024Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restorationcited by 34
  12. 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32
38 more
  1. 2024SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flowcited by 30
  2. 2024Hierarchical Patch Diffusion Models for High-Resolution Video Generationcited by 24
  3. 2024Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediatorscited by 24
  4. 2024LaVin-DiT: Large Vision Diffusion Transformercited by 21
  5. 2024Alfie: Democratising RGBA Image Generation With No cited by 20
  6. 2025Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Lightcited by 18
  7. 2025DiffLocks: Generating 3D Hair from a Single Image using Diffusion Modelscited by 16
  8. 2025Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformercited by 15
  9. 2024Fast LiDAR Data Generation with Rectified Flowscited by 14
  10. 2024DiC: Rethinking Conv3x3 Designs in Diffusion Modelscited by 14
  11. 2024Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Datacited by 13
  12. 2025Adapting Self-Supervised Representations as a Latent Space for Efficient Generationcited by 13
  13. 2025Effective Cloud Removal for Remote Sensing Images by an Improved Mean-Reverting Denoising Model with Elucidated Design Spacecited by 12
  14. 2025InvFusion: Bridging Supervised and Zero-shot Diffusion for Inverse Problemscited by 10
  15. 2025Scaling Up AI-Generated Image Detection with Generator-Aware Prototypescited by 9
  16. 2024Phy-Diff: Physics-guided Hourglass Diffusion Model for Diffusion MRI Synthesiscited by 8
  17. 2024VariFace: Fair and Diverse Synthetic Dataset Generation for Face Recognitioncited by 8
  18. 2025MirrorVerse: Pushing Diffusion Models to Realistically Reflect the Worldcited by 8
  19. 2025VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Modelscited by 8
  20. 2026Asymmetric Flow Modelscited by 8
  21. 2024Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformerscited by 7
  22. 2025SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenizationcited by 7
  23. 2026ReHyAt: Recurrent Hybrid Attention for Video Diffusion Transformerscited by 7
  24. 2026S2DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generationcited by 7
  25. 2026Low Light Image Enhancement Challenge at NTIRE 2026cited by 7
  26. 2026MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformercited by 7
  27. 2025Scaling Diffusion Transformers Efficiently via μPcited by 6
  28. 2025Trainable Log-linear Sparse Attention for Efficient Diffusion Transformerscited by 6
  29. 2024Cottention: Linear Transformers With Cosine Attentioncited by 5
  30. 2024EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketchingcited by 5
  31. 2024Efficient Scaling of Diffusion Transformers for Text-to-Image Generationcited by 5
  32. 2026AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantizationcited by 5
  33. 2024Cross-modality image synthesis from TOF-MRA to CTA using diffusion-based modelscited by 4
  34. 2024CryoFM: A Flow-based Foundation Model for Cryo-EM Densitiescited by 4
  35. 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigmcited by 4
  36. 2025DiTVR: Zero-Shot Diffusion Transformer for Video Restorationcited by 4
  37. 2025Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentialscited by 4
  38. 2026Prior-Informed Flow Matching for Graph Reconstructioncited by 4

50 most cited of 118 All 118 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models
  2. 2025PixelDiT: Pixel Diffusion Transformers for Image Generation
  3. 2024Greedy Growing Enables High-Resolution Pixel-Based Diffusion Models
  4. 2022Exploring Transformer Backbones for Image Diffusion Models
  5. 2025DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion
  6. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis
  7. 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources
  8. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance