# A Deep and Tractable Density Estimator

arXiv: https://arxiv.org/abs/1310.1757 · HTML: https://mlsanity.com/paper/1310.1757/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2013.

## Built on by (most cited first, 30 of 68)

- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024)
- [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://mlsanity.com/paper/2409.02908/) (2024)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking](https://mlsanity.com/paper/2505.24857/) (2025)
- [SceneGen: Learning to Generate Realistic Traffic Scenes](https://mlsanity.com/paper/2101.06541/) (2021)
- [Path Planning for Masked Diffusion Model Sampling](https://mlsanity.com/paper/2502.03540/) (2025)
- [Training and Inference on Any-Order Autoregressive Models the Right Way](https://mlsanity.com/paper/2205.13554/) (2022)
- [GraphGDP: Generative Diffusion Processes for Permutation Invariant Graph Generation](https://mlsanity.com/paper/2212.01842/) (2022)
- [TACTiS: Transformer-Attentional Copulas for Time Series](https://mlsanity.com/paper/2202.03528/) (2022)
- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Esoteric Language Models: A Family of Any-Order Diffusion LLMs](https://mlsanity.com/paper/2506.01928/) (2025)
- [Autoregressive Conditional Neural Processes](https://mlsanity.com/paper/2303.14468/) (2023)
- [Regression as Classification: Influence of Task Formulation on Neural Network Features](https://mlsanity.com/paper/2211.05641/) (2022)
- [Learning-Order Autoregressive Models with Application to Molecular Graph Generation](https://mlsanity.com/paper/2503.05979/) (2025)
- [Latent Thought Models with Variational Bayes Inference-Time Computation](https://mlsanity.com/paper/2502.01567/) (2025)
- [Set Block Decoding is a Language Model Inference Accelerator](https://mlsanity.com/paper/2509.04185/) (2025)
- [Arbitrary Conditional Distributions with Energy](https://mlsanity.com/paper/2102.04426/) (2021)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding](https://mlsanity.com/paper/2504.20456/) (2025)
- [Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach](https://mlsanity.com/paper/2108.05094/) (2021)
- [Planner Aware Path Learning in Diffusion Language Models Training](https://mlsanity.com/paper/2509.23405/) (2025)
- [Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations](https://mlsanity.com/paper/2510.25544/) (2025)
- [Self-Speculative Masked Diffusions](https://mlsanity.com/paper/2510.03929/) (2025)
- [Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo](https://mlsanity.com/paper/2502.06379/) (2025)
- [dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning](https://mlsanity.com/paper/2512.21446/) (2025)
- [DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking](https://mlsanity.com/paper/2603.01367/) (2026)

## Abstract (arXiv)

The Neural Autoregressive Distribution Estimator (NADE) and its real-valued version RNADE are competitive density models of multidimensional data across a variety of domains. These models use a fixed, arbitrary ordering of the data dimensions. One can easily condition on variables at the beginning of the ordering, and marginalize out variables at the end of the ordering, however other inference tasks require approximate inference. In this work we introduce an efficient procedure to simultaneously train a NADE model for each possible ordering of the variables, by sharing parameters across all these models. We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available. Moreover, unlike the original NADE, our training procedure scales to deep models. Empirically, ensembles of Deep NADE models obtain state of the art density estimation performance.