# Subspace Diffusion Generative Models arXiv: https://arxiv.org/abs/2205.01490 · HTML: https://mlsanity.com/paper/2205.01490/ ## Key facts - Cited by 70 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 20 of its 22 references that are in the map. - First posted on arXiv 3 May 2022. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 20 of 20) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021) - [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019) - [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021) - [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021) - [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021) - [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022) - [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021) - [How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks](https://mlsanity.com/paper/2009.11848/) (2020) - [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021) - [On Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.00132/) (2021) - [Learning to Efficiently Sample from Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.03802/) (2021) - [Noise Estimation for Generative Diffusion Models](https://mlsanity.com/paper/2104.02600/) (2021) - [Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling](https://mlsanity.com/paper/1812.01608/) (2018) - [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022) ## Built on by (most cited first, 30 of 70) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022) - [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022) - [Torsional Diffusion for Molecular Conformer Generation](https://mlsanity.com/paper/2206.01729/) (2022) - [Structural Pruning for Diffusion Models](https://mlsanity.com/paper/2305.10924/) (2023) - [Convergence of score-based generative modeling for general data distributions](https://mlsanity.com/paper/2209.12381/) (2022) - [Convergence for score-based generative modeling with polynomial complexity](https://mlsanity.com/paper/2206.06227/) (2022) - [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022) - [DiffCollage: Parallel Generation of Large Content with Diffusion Models](https://mlsanity.com/paper/2303.17076/) (2023) - [BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping](https://mlsanity.com/paper/2306.05544/) (2023) - [Blurring Diffusion Models](https://mlsanity.com/paper/2209.05557/) (2022) - [Reflected Diffusion Models](https://mlsanity.com/paper/2304.04740/) (2023) - [Score-based Diffusion Models in Function Space](https://mlsanity.com/paper/2302.07400/) (2023) - [Diffusion Models, Image Super-Resolution And Everything: A Survey](https://mlsanity.com/paper/2401.00736/) (2024) - [Wavelet Score-Based Generative Modeling](https://mlsanity.com/paper/2208.05003/) (2022) - [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023) - [PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model](https://mlsanity.com/paper/2306.02531/) (2023) - [Efficient Diffusion Models for Vision: A Survey](https://mlsanity.com/paper/2210.09292/) (2022) - [OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Models](https://mlsanity.com/paper/2306.08860/) (2023) - [Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport](https://mlsanity.com/paper/2305.14777/) (2023) - [Few-Shot Diffusion Models](https://mlsanity.com/paper/2205.15463/) (2022) - [Fast Graph Generation via Spectral Diffusion](https://mlsanity.com/paper/2211.08892/) (2022) - [Trans-Dimensional Generative Modeling via Jump Diffusion Models](https://mlsanity.com/paper/2305.16261/) (2023) - [Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis](https://mlsanity.com/paper/2207.11192/) (2022) - [f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation](https://mlsanity.com/paper/2210.04955/) (2022) - [Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors](https://mlsanity.com/paper/2403.11407/) (2024) - [Spectral Diffusion Processes](https://mlsanity.com/paper/2209.14125/) (2022) - [Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport](https://mlsanity.com/paper/2402.05443/) (2024) - [Pyramidal Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2208.01864/) (2022) - [∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified States](https://mlsanity.com/paper/2303.18242/) (2023) ## Abstract (arXiv) Score-based models generate samples by mapping noise to data (and vice versa) via a high-dimensional diffusion process. We question whether it is necessary to run this entire process at high dimensionality and incur all the inconveniences thereof. Instead, we restrict the diffusion via projections onto subspaces as the data distribution evolves toward noise. When applied to state-of-the-art models, our framework simultaneously improves sample quality -- reaching an FID of 2.17 on unconditional CIFAR-10 -- and reduces the computational cost of inference for the same number of denoising steps. Our framework is fully compatible with continuous-time diffusion and retains its flexible capabilities, including exact log-likelihoods and controllable generation. Code is available at https://github.com/bjing2016/subspace-diffusion.