mlsanityData to 7 Oct 2026

Pixel-Space Diffusion Transformers

arXiv 2607.17585 ↗First posted 20 Jul 2026cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Latent diffusion models (LDMs) enable efficient high-resolution image synthesis by denoising in a VAE-compressed latent space. However, fixed visual tokenizers can discard fine textures and structural details, while separate representation and diffusion training creates a mismatch between reconstruction and generation objectives. These limitations have renewed interest in pixel-space diffusion, which models raw pixels directly, removes the VAE bottleneck, and supports end-to-end optimization. This formulation better matches the demands of high-fidelity generation but introduces challenges in high-dimensional modeling, including noise scheduling, loss weighting, token efficiency, and scalable architecture design. Pixel-space modeling also offers a promising basis for unified multimodal systems: raw pixels, text, and task conditions can be represented in a shared token space and jointly processed by a single Transformer, narrowing the gap between visual understanding and generation. This paper reviews Pixel-Space Diffusion Transformers (pDiTs) from the perspectives of model architecture, continuous generative mechanisms, and unified multimodal modeling. We summarize representative methods, identify key technical challenges, and discuss future directions toward high-fidelity, end-to-end vision foundation models that integrate generation and understanding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 112 of its 139 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

112 of 139 references.

Closest ancestors · 36

Same direction, newest first

  1. 2026Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis20 May
  2. 2026HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion15 May
  3. 2026Representation Alignment for Just Image Transformers is not Easier than You Think15 Mar
  4. 2026Scale Space Diffusion9 Mar
  5. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation11 Feb
  6. 2025Your Latent Mask is Wrong: Pixel-Equivalent Latent Compositing for Diffusion Models4 Dec
  7. 2025PixelDiT: Pixel Diffusion Transformers for Image Generation25 Nov
  8. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation24 Nov
  9. 2025DiP: Taming Diffusion Models in Pixel Space24 Nov
  10. 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models21 Oct
  11. 2025PixNerd: Pixel Neural Field Diffusion31 Jul
  12. 2025Exploring Diffusion Transformer Designs via Grafting5 Jun
24 more
  1. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training23 May
  2. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers14 Apr
  3. 2025U-REPA: Aligning Diffusion U-Nets to ViTs24 Mar
  4. 2025LeanVAE: An Ultra-Efficient Reconstruction VAE for Video Diffusion Models18 Mar
  5. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models2 Jan
  6. 2024One Step Diffusion via Shortcut Models16 Oct
  7. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  8. 2024Pixel-Space Post-Training of Latent Diffusion Models26 Sep
  9. 2024Diffusion Models and Representation Learning: A Survey30 Jun
  10. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  12. 2024Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers21 Jan
  13. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  14. 2023FreeU: Free Lunch in Diffusion U-Net20 Sep
  15. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  16. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  17. 2022Scalable Diffusion Models with Transformers19 Dec
  18. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  19. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  20. 2022Diffusion Models in Vision: A Survey10 Sep
  21. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  22. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  23. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  24. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 76

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
38 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2025DINOv3cited by 1,297
  6. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  7. 2022Human Motion Diffusion Modelcited by 1,062
  8. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  9. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  10. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  11. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  12. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  13. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  14. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  15. 2024OmniGen: Unified Image Generationcited by 388
  16. 2016Ladder Variational Autoencoderscited by 315
  17. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  18. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  19. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  20. 2019Pareto Multi-Task Learningcited by 224
  21. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  22. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  23. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 174
  24. 2016Energy-based Generative Adversarial Networkcited by 119
  25. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  26. 2022Latent Diffusion for Language Generationcited by 110
  27. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  28. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 91
  29. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 84
  30. 2020SmaAt-UNet: Precipitation Nowcasting using a Small Attention-UNet Architecturecited by 71
  31. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  32. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  33. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  34. 2023MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Imagescited by 41
  35. 2026HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformercited by 29
  36. 2020Longitudinal Variational Autoencodercited by 25
  37. 2024Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocationcited by 23
  38. 2024Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Modelscited by 22

50 most cited of 76 All 76 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PixelDiT2: Representation-Grounded Pixel Diffusion Transformers
  2. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
  3. 2026A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples
  4. 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models
  5. 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
  6. 2026FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion
  7. 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources
  8. 2026DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer