# PixelDiT: Pixel Diffusion Transformers for Image Generation

arXiv: https://arxiv.org/abs/2511.20645 · HTML: https://mlsanity.com/paper/2511.20645/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 43 of its 52 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 43)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Analyzing and Improving the Training Dynamics of Diffusion Models](https://mlsanity.com/paper/2312.02696/) (2023)
- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation](https://mlsanity.com/paper/2303.00848/) (2023)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation](https://mlsanity.com/paper/2402.17245/) (2024)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)
- [Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models](https://mlsanity.com/paper/2410.10733/) (2024)
- [FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models](https://mlsanity.com/paper/2412.08629/) (2024)

## Built on by (most cited first, 30 of 83)

- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [One-step Latent-free Image Generation with Pixel Mean Flows](https://mlsanity.com/paper/2601.22158/) (2026)
- [SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture](https://mlsanity.com/paper/2605.12500/) (2026)
- [Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation](https://mlsanity.com/paper/2604.24763/) (2026)
- [PixelGen: Improving Pixel Diffusion with Perceptual Supervision](https://mlsanity.com/paper/2602.02493/) (2026)
- [L2P: Unlocking Latent Potential for Pixel Generation](https://mlsanity.com/paper/2605.12013/) (2026)
- [HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion](https://mlsanity.com/paper/2605.15741/) (2026)
- [Image Generation with a Sphere Encoder](https://mlsanity.com/paper/2602.15030/) (2026)
- [Asymmetric Flow Models](https://mlsanity.com/paper/2605.12964/) (2026)
- [PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion](https://mlsanity.com/paper/2606.27760/) (2026)
- [Laminating Representation Autoencoders for Efficient Diffusion](https://mlsanity.com/paper/2602.04873/) (2026)
- [PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion](https://mlsanity.com/paper/2605.23902/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [End-to-End Training for Unified Tokenization and Latent Denoising](https://mlsanity.com/paper/2603.22283/) (2026)
- [An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models](https://mlsanity.com/paper/2608.16887/) (2026)
- [Spectrally-Guided Diffusion Noise Schedules](https://mlsanity.com/paper/2603.19222/) (2026)
- [FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion](https://mlsanity.com/paper/2605.17759/) (2026)
- [PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space](https://mlsanity.com/paper/2607.05373/) (2026)
- [Three-Body Scattering for Generative Modeling](https://mlsanity.com/paper/2607.18198/) (2026)
- [NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces](https://mlsanity.com/paper/2603.00180/) (2026)
- [FREPix: Frequency-Heterogeneous Flow Matching for Pixel-Space Image Generation](https://mlsanity.com/paper/2605.06421/) (2026)
- [Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers](https://mlsanity.com/paper/2607.28611/) (2026)
- [WaiT for the Signal: Simple Frequency-Aware Flow-Matching](https://mlsanity.com/paper/2607.28760/) (2026)
- [CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization](https://mlsanity.com/paper/2608.02502/) (2026)
- [V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising](https://mlsanity.com/paper/2603.16792/) (2026)
- [Registers Matter for Pixel-Space Diffusion Transformers](https://mlsanity.com/paper/2605.16147/) (2026)
- [RiT: Vanilla Diffusion Transformers Suffice in Representation Space](https://mlsanity.com/paper/2605.21981/) (2026)
- [CrossFlow: One-Step Generation Across Latent and Pixel Spaces](https://mlsanity.com/paper/2606.19970/) (2026)
- [PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation](https://mlsanity.com/paper/2607.01803/) (2026)
- [SPEED: One-Step Pixel Diffusion for High-quality Video Frame Interpolation](https://mlsanity.com/paper/2607.15585/) (2026)

## Abstract (arXiv)

Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstruction, leading to error accumulation while hindering joint optimization. To address these issues, we propose PixelDiT, a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space. PixelDiT adopts a fully transformer-based architecture shaped by a dual-level design: a patch-level DiT that captures global semantics and a pixel-level DiT that refines texture details, enabling efficient training of a pixel-space diffusion model while preserving fine details. PixelDiT achieves 1.61 FID on ImageNet 256 and 1.81 FID on ImageNet 512, surpassing existing pixel generative models. We further extend PixelDiT to text-to-image generation and pretrain it at the 10242resolution in pixel space. It achieves 0.74 on GenEval and 83.5 on DPG-bench, approaching the best latent diffusion models. Code: https://github.com/NVlabs/PixelDiT