mlsanityData to 14 Oct 2026

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers

arXiv 2609.24919 ↗First posted 21 Sep 2026cs.CVcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag behind in final image quality. We argue that a key reason is the lack of an explicit representation prior: unlike latent diffusion, which usually denoises in a compact and structured latent space, pixel diffusion needs to learn denoising-friendly representations and pixel generation simultaneously from raw RGB space. To address this problem, we propose PixelDiT2, an end-to-end pixel-space diffusion model designed to decouple representation learning from pixel generation without introducing an autoencoder or latent reconstruction bottleneck. We propose representation grounding that uses a frozen pretrained vision foundation model to provide explicit per-patch representation guidance throughout denoising, allowing the pixel diffusion transformer to focus more on pixel generation. On ImageNet-256x256, PixelDiT2 achieves an FID of 1.46 after 600 epochs; at 512x512 resolution, PixelDiT2 achieves an FID of 1.48 after 680 epochs. Project page: https://pixeldit.github.io/pixeldit2/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 49 of its 55 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

49 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration11 Apr

Other references · 48

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2022Flow Matching for Generative Modelingcited by 5,702
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
36 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2016Improved Techniques for Training GANscited by 2,970
  3. 2025DINOv3cited by 1,414
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  5. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  6. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  7. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  8. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  9. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  10. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 422
  11. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  12. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 304
  13. 2025Diffusion Transformers with Representation Autoencoderscited by 295
  14. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 262
  15. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 254
  16. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  17. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  18. 2023Fast Training of Diffusion Models with Masked Transformerscited by 176
  19. 2025DDT: Decoupled Diffusion Transformercited by 109
  20. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
  21. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 97
  22. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 83
  23. 2025PixNerd: Pixel Neural Field Diffusioncited by 82
  24. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusioncited by 75
  25. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  26. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generationcited by 60
  27. 2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 46
  28. 2025Fractal Generative Modelscited by 46
  29. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 36
  30. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 24
  31. 2025FARMER: Flow AutoRegressive Transformer over Pixelscited by 18
  32. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 18
  33. 2025SimFlow: Simplified and End-to-End Training of Latent Normalizing Flowscited by 7
  34. 2026RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editingcited by 7
  35. 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Modelscited by 6
  36. 2026GenFirst: Generation Before Reconstruction for Stable End-to-End Latent Generative Modelingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Pixel-Space Diffusion Transformers
  2. 2026A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples
  3. 2026PixelGen: Improving Pixel Diffusion with Perceptual Supervision
  4. 2025DiP: Taming Diffusion Models in Pixel Space
  5. 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
  6. 2026Learning Representations to Guide Pixel Diffusion Transformers
  7. 2024Pixel-Space Post-Training of Latent Diffusion Models
  8. 2026PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion