# Dimension-independent rates for structured neural density estimation

arXiv: https://arxiv.org/abs/2411.15095 · HTML: https://mlsanity.com/paper/2411.15095/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Causal Autoregressive Flows](https://mlsanity.com/paper/2011.02268/) (2020)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions](https://mlsanity.com/paper/2402.15602/) (2024)
- [Graphical Normalizing Flows](https://mlsanity.com/paper/2006.02548/) (2020)
- [How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View](https://mlsanity.com/paper/1712.08244/) (2017)
- [Structured Neural Networks for Density Estimation and Causal Inference](https://mlsanity.com/paper/2311.02221/) (2023)
- [Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions](https://mlsanity.com/paper/2402.08082/) (2024)
- [Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality](https://mlsanity.com/paper/1907.02177/) (2019)

## Built on by (most cited first, 5 of 5)

- [Statistical Properties of Rectified Flow](https://mlsanity.com/paper/2511.03193/) (2025)
- [Nonparametric estimation of a factorizable density using diffusion models](https://mlsanity.com/paper/2501.01783/) (2025)
- [PMODE: Theoretically Grounded and Modular Mixture Modeling](https://mlsanity.com/paper/2508.21396/) (2025)
- [CINDES: Classification induced neural density estimator and simulator](https://mlsanity.com/paper/2510.00367/) (2025)
- [Sample complexity bounds for categorical Markov random fields via Discrete Diffusions](https://mlsanity.com/paper/2610.02128/) (2026)

## Abstract (arXiv)

We show that deep neural networks achieve dimension-independent rates of convergence for learning structured densities such as those arising in image, audio, video, and text applications. More precisely, we demonstrate that neural networks with a simple L2-minimizing loss achieve a rate of n-1/(4+r) in nonparametric density estimation when the underlying density is Markov to a graph whose maximum clique size is at most r, and we provide evidence that in the aforementioned applications, this size is typically constant, i.e., r=O(1). We then establish that the optimal rate in L1 is n-1/(2+r) which, compared to the standard nonparametric rate of n-1/(2+d), reveals that the effective dimension of such problems is the size of the largest clique in the Markov random field. These rates are independent of the data's ambient dimension, making them applicable to realistic models of image, sound, video, and text data. Our results provide a novel justification for deep learning's ability to circumvent the curse of dimensionality, demonstrating dimension-independent convergence rates in these contexts.