# GUD: Generation with Unified Diffusion

arXiv: https://arxiv.org/abs/2410.02667 · HTML: https://mlsanity.com/paper/2410.02667/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 22 of 22)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Generative Modelling With Inverse Heat Dissipation](https://mlsanity.com/paper/2206.13397/) (2022)
- [Rolling Diffusion Models](https://mlsanity.com/paper/2402.09470/) (2024)
- [Blurring Diffusion Models](https://mlsanity.com/paper/2209.05557/) (2022)
- [Wavelet Score-Based Generative Modeling](https://mlsanity.com/paper/2208.05003/) (2022)
- [Mutual Information, Neural Networks and the Renormalization Group](https://mlsanity.com/paper/1704.06279/) (2017)
- [Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models](https://mlsanity.com/paper/2210.12254/) (2022)
- [Bayesian Renormalization](https://mlsanity.com/paper/2305.10491/) (2023)
- [Image generation with shortest path diffusion](https://mlsanity.com/paper/2306.00501/) (2023)
- [Sequential Data Generation with Groupwise Diffusion Process](https://mlsanity.com/paper/2310.01400/) (2023)

## Built on by (most cited first, 4 of 4)

- [A Fourier Space Perspective on Diffusion Models](https://mlsanity.com/paper/2505.11278/) (2025)
- [Exploiting the Exact Denoising Posterior Score in Training-Free Guidance of Diffusion Models](https://mlsanity.com/paper/2506.13614/) (2025)
- [Low-Pass Flow Matching](https://mlsanity.com/paper/2606.02177/) (2026)
- [AsyncPatch Diffusion: spatially-flexible image generation](https://mlsanity.com/paper/2606.07079/) (2026)

## Abstract (arXiv)

Diffusion generative models transform noise into data by inverting a process that progressively adds noise to data samples. Inspired by concepts from the renormalization group in physics, which analyzes systems across different scales, we revisit diffusion models by exploring three key design aspects: 1) the choice of representation in which the diffusion process operates (e.g. pixel-, PCA-, Fourier-, or wavelet-basis), 2) the prior distribution that data is transformed into during diffusion (e.g. Gaussian with covariance Sigma), and 3) the scheduling of noise levels applied separately to different parts of the data, captured by a component-wise noise schedule. Incorporating the flexibility in these choices, we develop a unified framework for diffusion generative models with greatly enhanced design freedom. In particular, we introduce soft-conditioning models that smoothly interpolate between standard diffusion models and autoregressive models (in any basis), conceptually bridging these two approaches. Our framework opens up a wide design space which may lead to more efficient training and data generation, and paves the way to novel architectures integrating different generative approaches and generation tasks.