mlsanityData to 14 Oct 2026

DiP: Taming Diffusion Models in Pixel Space

arXiv 2511.18822 ↗First posted 24 Nov 2025cs.CVcited by 48

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from potential information loss and non-end-to-end training. In contrast, existing pixel space models bypass VAEs but are computationally prohibitive for high-resolution synthesis. To resolve this dilemma, we propose DiP, an efficient pixel space diffusion framework. DiP decouples generation into a global and a local stage: a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details. This synergistic design achieves computational efficiency comparable to LDMs without relying on a VAE. DiP is accomplished with up to 10× faster inference speeds than previous method while increasing the total number of parameters by only 0.3%, and achieves an 1.79 FID score on ImageNet 256×256.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 63 of its 74 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

63 of 74 references.

Closest ancestors · 29

Same direction, newest first

  1. 2025PixNerd: Pixel Neural Field Diffusion31 Jul
  2. 2025DDT: Decoupled Diffusion Transformer8 Apr
  3. 2025Improving the Diffusability of Autoencoders20 Feb
  4. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models2 Jan
  5. 2024On Inductive Biases That Enable Generalization of Diffusion Transformers28 Oct
  6. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion25 Oct
  7. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models14 Oct
  8. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers14 Oct
  9. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  10. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  12. 2024Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement15 Feb
17 more
  1. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  2. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  4. 2023Relay Diffusion: Unifying diffusion process across resolutions for image synthesis4 Sep
  5. 2023Patched Denoising Diffusion Models For High-Resolution Image Synthesis2 Aug
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  7. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  8. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  9. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  10. 2022Scalable Diffusion Models with Transformers19 Dec
  11. 2022Classifier-Free Diffusion Guidance26 Jul
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  13. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  14. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  15. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  16. 2020Denoising Diffusion Implicit Models6 Oct
  17. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2016Improved Techniques for Training GANscited by 2,970
  11. 2014Conditional Generative Adversarial Netscited by 2,162
  12. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
22 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  2. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  3. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  4. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  5. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  6. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  7. 2021Generating Images with Sparse Representationscited by 266
  8. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  9. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
  10. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 97
  11. 2023Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restorationcited by 88
  12. 2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 46
  13. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 37
  14. 2023Diffusion Model for Camouflaged Object Detectioncited by 27
  15. 2025UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Datasetcited by 18
  16. 2025FARMER: Flow AutoRegressive Transformer over Pixelscited by 18
  17. 2024InstanceCap: Improving Text-to-Video Generation via Instance-aware Structured Captioncited by 17
  18. 2024FlowDCN: Exploring DCN-like Architectures for Fast Image Generation with Arbitrary Resolutioncited by 16
  19. 2024TAR3D: Creating High-Quality 3D Assets via Next-Part Predictioncited by 16
  20. 2025AR-1-to-3: Single Image to Consistent 3D Object Generation via Next-View Predictioncited by 15
  21. 2023TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshescited by 14
  22. 2024Computational Tradeoffs in Image Synthesis: Diffusion, Masked-Token, and Next-Token Predictioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 48.

48 papers.

Sort:

By area:

  1. 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 53
  2. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 32
  3. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 18
  4. 2026LUVE : Latent-Cascaded Ultra-High-Resolution Video Generation with Dual Frequency Expertscited by 15
  5. 2026PixelGen: Improving Pixel Diffusion with Perceptual Supervisioncited by 14
  6. 2026L2P: Unlocking Latent Potential for Pixel Generationcited by 9
  7. 2026HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusioncited by 9
  8. 2026Asymmetric Flow Modelscited by 8
  9. 2026PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusioncited by 8
  10. 2026From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmarkcited by 7
  11. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 7
  12. 2026End-to-End Training for Unified Tokenization and Latent Denoisingcited by 6
36 more
  1. 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Modelscited by 6
  2. 2026RefineAnything: Multimodal Region-Specific Refinement for Perfect Local Detailscited by 5
  3. 2026FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusioncited by 4
  4. 2026PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Spacecited by 4
  5. 2026Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioningcited by 3
  6. 2026WaiT for the Signal: Simple Frequency-Aware Flow-Matchingcited by 3
  7. 2026V-Co: A Closer Look at Visual Representation Alignment via Co-Denoisingcited by 2
  8. 2026JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curationcited by 2
  9. 2026OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generationcited by 2
  10. 2026SPEED: One-Step Pixel Diffusion for High-quality Video Frame Interpolationcited by 2
  11. 2026A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samplescited by 2
  12. 2026Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusioncited by 2
  13. 2026PixVerve: Advancing Native UHR Image Generation to 100MP with a Large-Scale High-Quality Datasetcited by 1
  14. 2026VINS-120K: Ultra High-Resolution Image Editing with A Large-Scale Datasetcited by 1
  15. 2026PixIE: Prompted Pixel-Space Low-Light Image Enhancementcited by 1
  16. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  17. 2026BareWave: Waveform-Native Flow-Matching Text-to-Speechcited by 1
  18. 2026Pixel-Space Diffusion Transformerscited by 1
  19. 2026Generative Video Compression with Adaptive Score Distillationcited by 1
  20. 2026PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimationcited by 1
  21. 2026Pixel-Space Diffusion via Observation Operatorscited by 1
  22. 2026Balancing Frequencies and Pixels in Flow Matchingcited by 1
  23. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 1
  24. 2026UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglementcited by 0
  25. 2026RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representationscited by 0
  26. 2026MOSAIK: Multi-Patch Content-Aware Spatial Allocation of Image Tokens for Efficient Generationcited by 0
  27. 2026Energy-Guided Flow Matchingcited by 0
  28. 2026SynVAR: Synergizing Spatial and Semantic Alignment in Visual Autoregressive Modelcited by 0
  29. 2026Advanced Pixel Diffusion Model with Guided Sparse Global Refinementcited by 0
  30. 2026Residual-Stream Burden Shapes Representation Learning in Diffusion Transformerscited by 0
  31. 2026Adversarial Training for Pixel Diffusioncited by 0
  32. 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuningcited by 0
  33. 2026Learning Representations to Guide Pixel Diffusion Transformerscited by 0
  34. 2026CoVA: Composing VAE Representations for Generative Learningcited by 0
  35. 2026Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generationcited by 0
  36. 2026PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation
  2. 2025PixelDiT: Pixel Diffusion Transformers for Image Generation
  3. 2026PixelDiT2: Representation-Grounded Pixel Diffusion Transformers
  4. 2025One Small Step in Latent, One Giant Leap for Pixels: Fast Latent Upscale Adapter for Your Diffusion Models
  5. 2026DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer
  6. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  7. 2024Diffusion Models Need Visual Priors for Image Generation
  8. 2023Diffusion Models Without Attention