# Back to Basics: Let Denoising Generative Models Denoise

arXiv: https://arxiv.org/abs/2511.13720 · HTML: https://mlsanity.com/paper/2511.13720/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- First posted on arXiv 17 Nov 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Built on by (most cited first, 16 of 16)

- [Flex-π: A Multi-Stream World-Action Model with Compute Flexibility](https://mlsanity.com/paper/2608.10860/) (2026)
- [V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think](https://mlsanity.com/paper/2604.23380/) (2026)
- [Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling](https://mlsanity.com/paper/2604.28185/) (2026)
- [PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space](https://mlsanity.com/paper/2607.05373/) (2026)
- [LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching](https://mlsanity.com/paper/2606.10517/) (2026)
- [PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics](https://mlsanity.com/paper/2609.19142/) (2026)
- [The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL](https://mlsanity.com/paper/2606.19162/) (2026)
- [x-Prediction Is All You Need:Training-Free Accelerated Generation via Endpoint Decodability](https://mlsanity.com/paper/2607.06114/) (2026)
- [Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics](https://mlsanity.com/paper/2605.29351/) (2026)
- [Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation](https://mlsanity.com/paper/2610.00575/) (2026)
- [Xaurora: Generative Weather Forecasting with Denoising Stochastic Interpolants from a Foundation Model Prior](https://mlsanity.com/paper/2610.06509/) (2026)
- [S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation](https://mlsanity.com/paper/2610.06847/) (2026)
- [Feature Information Dynamics in Diffusion](https://mlsanity.com/paper/2610.08626/) (2026)
- [Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning](https://mlsanity.com/paper/2610.09450/) (2026)
- [IntactWorld: Joint World Modeling with Intact Features](https://mlsanity.com/paper/2610.11174/) (2026)
- [Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher](https://mlsanity.com/paper/2610.11479/) (2026)

## Abstract (arXiv)

Today's denoising diffusion models do not "denoise" in the classical sense, i.e., they do not directly predict clean images. Rather, the neural networks predict noise or a noised quantity. In this paper, we suggest that predicting clean data and predicting noised quantities are fundamentally different. According to the manifold assumption, natural data should lie on a low-dimensional manifold, whereas noised quantities do not. With this assumption, we advocate for models that directly predict clean data, which allows apparently under-capacity networks to operate effectively in very high-dimensional spaces. We show that simple, large-patch Transformers on pixels can be strong generative models: using no tokenizer, no pre-training, and no extra loss. Our approach is conceptually nothing more than "Just image Transformers", or JiT, as we call it. We report competitive results using JiT with large patch sizes of 16 and 32 on ImageNet at resolutions of 256 and 512, where predicting high-dimensional noised quantities can fail catastrophically. With our networks mapping back to the basics of the manifold, our research goes back to basics and pursues a self-contained paradigm for Transformer-based diffusion on raw natural data.