# Deconvolutional Density Network: Modeling Free-Form Conditional Distributions

arXiv: https://arxiv.org/abs/2105.14367 · HTML: https://mlsanity.com/paper/2105.14367/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 23 of 23)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [RNADE: The real-valued neural autoregressive density-estimator](https://mlsanity.com/paper/1306.0186/) (2013)
- [Conditional Density Estimation with Bayesian Normalising Flows](https://mlsanity.com/paper/1802.04908/) (2018)
- [Uncertainty in the Variational Information Bottleneck](https://mlsanity.com/paper/1807.00906/) (2018)
- [Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks](https://mlsanity.com/paper/1903.00954/) (2019)
- [Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures](https://mlsanity.com/paper/1905.08550/) (2019)
- [Gaussian Process Conditional Density Estimation](https://mlsanity.com/paper/1810.12750/) (2018)
- [Noise Regularization for Conditional Density Estimation](https://mlsanity.com/paper/1907.08982/) (2019)
- [Arbitrary Conditional Distributions with Energy](https://mlsanity.com/paper/2102.04426/) (2021)
- [Deep Distribution Regression](https://mlsanity.com/paper/1903.06023/) (2019)
- [The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables](https://mlsanity.com/paper/1705.07111/) (2017)
- [Neural-Kernelized Conditional Density Estimation](https://mlsanity.com/paper/1806.01754/) (2018)
- [A Method to Model Conditional Distributions with Normalizing Flows](https://mlsanity.com/paper/1911.02052/) (2019)
- [Better Conditional Density Estimation for Neural Networks](https://mlsanity.com/paper/1606.02321/) (2016)

## Built on by (most cited first, 3 of 3)

- [Conformalized Deep Splines for Optimal and Efficient Prediction Sets](https://mlsanity.com/paper/2311.00774/) (2023)
- [Deep Modeling of Non-Gaussian Aleatoric Uncertainty](https://mlsanity.com/paper/2405.20513/) (2024)
- [Generative modeling of density regression through tree flows](https://mlsanity.com/paper/2406.05260/) (2024)

## Abstract (arXiv)

Conditional density estimation (CDE) is the task of estimating the probability of an event conditioned on some inputs. A neural network (NN) can also be used to compute the output distribution for continuous-domain, which can be viewed as an extension of regression task. Nevertheless, it is difficult to explicitly approximate a distribution without knowing the information of its general form a priori. In order to fit an arbitrary conditional distribution, discretizing the continuous domain into bins is an effective strategy, as long as we have sufficiently narrow bins and very large data. However, collecting enough data is often hard to reach and falls far short of that ideal in many circumstances, especially in multivariate CDE for the curse of dimensionality. In this paper, we demonstrate the benefits of modeling free-form conditional distributions using a deconvolution-based neural net framework, coping with data deficiency problems in discretization. It has the advantage of being flexible but also takes advantage of the hierarchical smoothness offered by the deconvolution layers. We compare our method to a number of other density-estimation approaches and show that our Deconvolutional Density Network (DDN) outperforms the competing methods on many univariate and multivariate tasks. The code of DDN is available at https://github.com/NBICLAB/DDN.