# There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-training

arXiv: https://arxiv.org/abs/2510.12586 · HTML: https://mlsanity.com/paper/2510.12586/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 44 of its 51 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025)
- [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024)
- [Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation](https://mlsanity.com/paper/2303.00848/) (2023)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)

## Built on by (most cited first, 24 of 24)

- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [One-step Latent-free Image Generation with Pixel Mean Flows](https://mlsanity.com/paper/2601.22158/) (2026)
- [DreamX-World 1.0: A General-Purpose Interactive World Model](https://mlsanity.com/paper/2606.16993/) (2026)
- [Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning](https://mlsanity.com/paper/2512.24146/) (2025)
- [PixelGen: Improving Pixel Diffusion with Perceptual Supervision](https://mlsanity.com/paper/2602.02493/) (2026)
- [ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraints](https://mlsanity.com/paper/2510.14847/) (2025)
- [MACE-Dance: Motion-Appearance Cascaded Experts for Music-Driven Dance Video Generation](https://mlsanity.com/paper/2512.18181/) (2025)
- [Asymmetric Flow Models](https://mlsanity.com/paper/2605.12964/) (2026)
- [Elucidating the SNR-t Bias of Diffusion Probabilistic Models](https://mlsanity.com/paper/2604.16044/) (2026)
- [OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data](https://mlsanity.com/paper/2606.30019/) (2026)
- [CrossFlow: One-Step Generation Across Latent and Pixel Spaces](https://mlsanity.com/paper/2606.19970/) (2026)
- [A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples](https://mlsanity.com/paper/2607.29122/) (2026)
- [Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion](https://mlsanity.com/paper/2609.36014/) (2026)
- [WiT: Waypoint Diffusion Transformers for Alleviating Trajectory Conflict in Pixel-Space Image Generation](https://mlsanity.com/paper/2603.15132/) (2026)
- [Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generation](https://mlsanity.com/paper/2606.00514/) (2026)
- [Pixel-Space Diffusion via Observation Operators](https://mlsanity.com/paper/2608.21885/) (2026)
- [Balancing Frequencies and Pixels in Flow Matching](https://mlsanity.com/paper/2609.02748/) (2026)
- [RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations](https://mlsanity.com/paper/2605.15908/) (2026)
- [Rethinking Pixel Mean Flows via Interval Denoiser](https://mlsanity.com/paper/2608.04818/) (2026)
- [DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution](https://mlsanity.com/paper/2608.31106/) (2026)
- [Genesis: A Generative Engine for Hierarchical Satellite Image Synthesis](https://mlsanity.com/paper/2609.02683/) (2026)
- [PixelDiT2: Representation-Grounded Pixel Diffusion Transformers](https://mlsanity.com/paper/2609.24919/) (2026)
- [PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion](https://mlsanity.com/paper/2610.00483/) (2026)
- [Learning Representations to Guide Pixel Diffusion Transformers](https://mlsanity.com/paper/2610.12978/) (2026)

## Abstract (arXiv)

Pixel-space generative models are often more difficult to train and generally underperform compared to their latent-space counterparts, leaving a persistent performance and efficiency gap. In this paper, we introduce a novel two-stage training framework that closes this gap for pixel-space diffusion and consistency models. In the first stage, we pre-train encoders to capture meaningful semantics from clean images while aligning them with points along the same deterministic sampling trajectory, which evolves points from the prior to the data distribution. In the second stage, we integrate the encoder with a randomly initialized decoder and fine-tune the complete model end-to-end for both diffusion and consistency models. Our framework achieves state-of-the-art (SOTA) performance on ImageNet. Specifically, our diffusion model reaches an FID of 1.58 on ImageNet-256 and 2.35 on ImageNet-512 with 75 number of function evaluations (NFE) surpassing prior pixel-space methods and VAE-based counterparts by a large margin in both generation quality and training efficiency. In a direct comparison, our model significantly outperforms DiT while using only around 30% of its training compute. Furthermore, our consistency model achieves an impressive FID of 8.82 on ImageNet-256, significantly outperforming its latent-space counterparts. This marks the first successful training of a consistency model directly on high-resolution images without relying on pre-trained VAEs or diffusion models. Our codes are available at: hrefhttps://github.com/AMAP-ML/EPGhttps://github.com/AMAP-ML/EPG