# Generative Modeling: A Review

arXiv: https://arxiv.org/abs/2501.05458 · HTML: https://mlsanity.com/paper/2501.05458/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 78 references that are in the map.
- First posted on arXiv 24 Dec 2024.

## Builds on (most cited first, 30 of 33)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Neural Importance Sampling](https://mlsanity.com/paper/1808.03856/) (2018)
- [Guided Image Generation with Conditional Invertible Neural Networks](https://mlsanity.com/paper/1907.02392/) (2019)
- [i-RevNet: Deep Invertible Networks](https://mlsanity.com/paper/1802.07088/) (2018)
- [Does data interpolation contradict statistical optimality?](https://mlsanity.com/paper/1806.09471/) (2018)
- [Understanding and Mitigating Exploding Inverses in Invertible Neural Networks](https://mlsanity.com/paper/2006.09347/) (2020)
- [Conditional Density Estimation with Bayesian Normalising Flows](https://mlsanity.com/paper/1802.04908/) (2018)
- [Posterior Concentration for Sparse Deep Learning](https://mlsanity.com/paper/1803.09138/) (2018)
- [Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem](https://mlsanity.com/paper/1906.11945/) (2019)
- [High-dimensional analysis of double descent for linear regression with random projections](https://mlsanity.com/paper/2303.01372/) (2023)
- [Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network](https://mlsanity.com/paper/1510.02175/) (2015)
- [MintNet: Building Invertible Neural Networks with Masked Convolutions](https://mlsanity.com/paper/1907.07945/) (2019)
- [Approximate Bayesian computation with the Wasserstein distance](https://mlsanity.com/paper/1905.03747/) (2019)
- [Autoregressive Quantile Networks for Generative Modeling](https://mlsanity.com/paper/1806.05575/) (2018)
- [Quantile regression with deep ReLU Networks: Estimators and minimax rates](https://mlsanity.com/paper/2010.08236/) (2020)
- [Deep Learning in Finance](https://mlsanity.com/paper/1602.06561/) (2016)
- [Towards a Theoretical Understanding of the Robustness of Variational Autoencoders](https://mlsanity.com/paper/2007.07365/) (2020)
- [Learning Summary Statistics for Bayesian Inference with Autoencoders](https://mlsanity.com/paper/2201.12059/) (2022)
- [Convolutional Neural Networks as Summary Statistics for Approximate Bayesian Computation](https://mlsanity.com/paper/2001.11760/) (2020)
- [Bayesian Indirect Inference Using a Parametric Auxiliary Model](https://mlsanity.com/paper/1505.03372/) (2015)

## Built on by (most cited first, 2 of 2)

- [Generative Learner for Distributional Causal Effects](https://mlsanity.com/paper/2503.17894/) (2025)
- [Generative Bayesian Hyperparameter Tuning](https://mlsanity.com/paper/2512.20051/) (2025)

## Abstract (arXiv)

We organize the generative-modeling literature around three classes of generators, corresponding to three distinct inferential tasks: estimating counterfactual outcome distributions in causal inference, recovering posteriors from simulated parameter--outcome pairs, and forming predictive outcome distributions. The unifying representation relies on the noise outsourcing theorem of Kallenberg, which expresses a conditional distribution as a deterministic function of its inputs and an independent noise variable. Within this organization we develop generative Bayesian computation, a method in the parameter--outcome class: a quantile neural network, trained on simulated pairs under the pinball loss, that targets the posterior of the parameter directly, without invertible architectures or density evaluation, and that serves equally as a predictive generator once the roles of parameter and outcome are exchanged. We illustrate the framework on an agent-based Ebola transmission application, where generative Bayesian computation recovers accurate posteriors at substantially lower cost than rejection-based simulation inference, while avoiding the density-evaluation and invertibility constraints of competing generators.