# Learning multi-scale local conditional probability models of images

arXiv: https://arxiv.org/abs/2303.02984 · HTML: https://mlsanity.com/paper/2303.02984/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 20 of 20)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [SNIPS: Solving Noisy Inverse Problems Stochastically](https://mlsanity.com/paper/2105.14951/) (2021)
- [Soft Diffusion: Score Matching for General Corruptions](https://mlsanity.com/paper/2209.05442/) (2022)
- [Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie](https://mlsanity.com/paper/2103.04715/) (2021)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser](https://mlsanity.com/paper/2007.13640/) (2020)
- [Wavelet Score-Based Generative Modeling](https://mlsanity.com/paper/2208.05003/) (2022)
- [Wavelet Flow: Fast Training of High Resolution Normalizing Flows](https://mlsanity.com/paper/2010.13821/) (2020)
- [Wavelet Conditional Renormalization Group](https://mlsanity.com/paper/2207.04941/) (2022)
- [Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks](https://mlsanity.com/paper/1712.07493/) (2017)
- [Learning Non-linear Wavelet Transformation via Normalizing Flow](https://mlsanity.com/paper/2101.11306/) (2021)

## Built on by (most cited first, 21 of 21)

- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data](https://mlsanity.com/paper/2402.16991/) (2024)
- [Matryoshka Diffusion Models](https://mlsanity.com/paper/2310.15111/) (2023)
- [Principal Uncertainty Quantification with Spatial Correlation for Image Restoration Problems](https://mlsanity.com/paper/2305.10124/) (2023)
- [Probing the Latent Hierarchical Structure of Data via Diffusion Models](https://mlsanity.com/paper/2410.13770/) (2024)
- [How Compositional Generalization and Creativity Improve as Diffusion Models are Trained](https://mlsanity.com/paper/2502.12089/) (2025)
- [DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generation](https://mlsanity.com/paper/2411.04168/) (2024)
- [Generalization Dynamics of Linear Diffusion Models](https://mlsanity.com/paper/2505.24769/) (2025)
- [TriNeRFLet: A Wavelet Based Triplane NeRF Representation](https://mlsanity.com/paper/2401.06191/) (2024)
- [MESSY Estimation: Maximum-Entropy based Stochastic and Symbolic densitY Estimation](https://mlsanity.com/paper/2306.04120/) (2023)
- [UDPM: Upsampling Diffusion Probabilistic Models](https://mlsanity.com/paper/2305.16269/) (2023)
- [Hierarchic Flows to Estimate and Sample High-dimensional Probabilities](https://mlsanity.com/paper/2405.03468/) (2024)
- [Conditionally Strongly Log-Concave Generative Models](https://mlsanity.com/paper/2306.00181/) (2023)
- [An analytic theory of convolutional neural network inverse problems solvers](https://mlsanity.com/paper/2601.10334/) (2026)
- [Image Statistics Predict the Sensitivity of Perceptual Quality Metrics](https://mlsanity.com/paper/2303.09874/) (2023)
- [Unconditional CNN denoisers contain sparse semantic representation of images](https://mlsanity.com/paper/2506.01912/) (2025)
- [Diffusion Transformer meets Multi-level Wavelet Spectrum for Single Image Super-Resolution](https://mlsanity.com/paper/2511.01175/) (2025)
- [Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation](https://mlsanity.com/paper/2607.28776/) (2026)
- [Renormalization Group Flow Matching for Scalable Local Generative Modeling](https://mlsanity.com/paper/2608.23696/) (2026)
- [Overcoming critical slowing down in frustrated spin systems by learned multiscale sampling](https://mlsanity.com/paper/2608.31114/) (2026)

## Abstract (arXiv)

Deep neural networks can learn powerful prior probability models for images, as evidenced by the high-quality generations obtained with recent score-based diffusion methods. But the means by which these networks capture complex global statistical structure, apparently without suffering from the curse of dimensionality, remain a mystery. To study this, we incorporate diffusion methods into a multi-scale decomposition, reducing dimensionality by assuming a stationary local Markov model for wavelet coefficients conditioned on coarser-scale coefficients. We instantiate this model using convolutional neural networks (CNNs) with local receptive fields, which enforce both the stationarity and Markov properties. Global structures are captured using a CNN with receptive fields covering the entire (but small) low-pass image. We test this model on a dataset of face images, which are highly non-stationary and contain large-scale geometric structures. Remarkably, denoising, super-resolution, and image synthesis results all demonstrate that these structures can be captured with significantly smaller conditioning neighborhoods than required by a Markov model implemented in the pixel domain. Our results show that score estimation for large complex images can be reduced to low-dimensional Markov conditional models across scales, alleviating the curse of dimensionality.