# High-Dimensional Probability Estimation with Deep Density Models

arXiv: https://arxiv.org/abs/1302.5125 · HTML: https://mlsanity.com/paper/1302.5125/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2013.

## Built on by (most cited first, 17 of 17)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding](https://mlsanity.com/paper/2102.11086/) (2021)
- [Invertible DenseNets with Concatenated LipSwish](https://mlsanity.com/paper/2102.02694/) (2021)
- [Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks](https://mlsanity.com/paper/2205.10041/) (2022)
- [Marginal Tail-Adaptive Normalizing Flows](https://mlsanity.com/paper/2206.10311/) (2022)
- [Deep Generative Models, Synthetic Tabular Data, and Differential Privacy: An Overview and Synthesis](https://mlsanity.com/paper/2307.15424/) (2023)
- [Deep Data Density Estimation through Donsker-Varadhan Representation](https://mlsanity.com/paper/2104.06612/) (2021)
- [Bootstrap Your Flow](https://mlsanity.com/paper/2111.11510/) (2021)
- [Generative Marginalization Models](https://mlsanity.com/paper/2310.12920/) (2023)
- [F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits](https://mlsanity.com/paper/2110.04253/) (2021)
- [Learning to Generate Training Datasets for Robust Semantic Segmentation](https://mlsanity.com/paper/2308.02535/) (2023)
- [Semi-Equivariant Continuous Normalizing Flows for Target-Aware Molecule Generation](https://mlsanity.com/paper/2211.04754/) (2022)
- [Probabilistic Forecasting of Irregular Time Series via Conditional Flows](https://mlsanity.com/paper/2402.06293/) (2024)
- [Semi-Equivariant Conditional Normalizing Flows](https://mlsanity.com/paper/2304.06779/) (2023)
- [Mutual Information Multinomial Estimation](https://mlsanity.com/paper/2408.09377/) (2024)
- [Calibrated Multivariate Regression with Localized PIT Mappings](https://mlsanity.com/paper/2409.10855/) (2024)

## Abstract (arXiv)

One of the fundamental problems in machine learning is the estimation of a probability distribution from data. Many techniques have been proposed to study the structure of data, most often building around the assumption that observations lie on a lower-dimensional manifold of high probability. It has been more difficult, however, to exploit this insight to build explicit, tractable density models for high-dimensional data. In this paper, we introduce the deep density model (DDM), a new approach to density estimation. We exploit insights from deep learning to construct a bijective map to a representation space, under which the transformation of the distribution of the data is approximately factorized and has identical and known marginal densities. The simplicity of the latent distribution under the model allows us to feasibly explore it, and the invertibility of the map to characterize contraction of measure across it. This enables us to compute normalized densities for out-of-sample data. This combination of tractability and flexibility allows us to tackle a variety of probabilistic tasks on high-dimensional datasets, including: rapid computation of normalized densities at test-time without evaluating a partition function; generation of samples without MCMC; and characterization of the joint entropy of the data.