# Flexible and accurate inference and learning for deep generative models

arXiv: https://arxiv.org/abs/1805.11051 · HTML: https://mlsanity.com/paper/1805.11051/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 9 of 9)

- [Active inference, Bayesian optimal design, and expected utility](https://mlsanity.com/paper/2110.04074/) (2021)
- [Unsupervised representation learning with recognition-parametrised probabilistic models](https://mlsanity.com/paper/2209.05661/) (2022)
- [Distributional Bellman Operators over Mean Embeddings](https://mlsanity.com/paper/2312.07358/) (2023)
- [Symmetric Equilibrium Learning of VAEs](https://mlsanity.com/paper/2307.09883/) (2023)
- [Expressive probabilistic sampling in recurrent neural networks](https://mlsanity.com/paper/2308.11809/) (2023)
- [Inferring Inference](https://mlsanity.com/paper/2310.03186/) (2023)
- [How does the brain compute with probabilities?](https://mlsanity.com/paper/2409.02709/) (2024)
- [A Unified Theory of Exact Inference and Learning in Exponential Family Latent Variable Models](https://mlsanity.com/paper/2404.19501/) (2024)
- [DRL-STAF: A Deep Reinforcement Learning Framework for State-Aware Forecasting of Complex Multivariate Hidden Markov Processes](https://mlsanity.com/paper/2605.14632/) (2026)

## Abstract (arXiv)

We introduce a new approach to learning in hierarchical latent-variable generative models called the "distributed distributional code Helmholtz machine", which emphasises flexibility and accuracy in the inferential process. In common with the original Helmholtz machine and later variational autoencoder algorithms (but unlike adverserial methods) our approach learns an explicit inference or "recognition" model to approximate the posterior distribution over the latent variables. Unlike in these earlier methods, the posterior representation is not limited to a narrow tractable parameterised form (nor is it represented by samples). To train the generative and recognition models we develop an extended wake-sleep algorithm inspired by the original Helmholtz Machine. This makes it possible to learn hierarchical latent models with both discrete and continuous variables, where an accurate posterior representation is essential. We demonstrate that the new algorithm outperforms current state-of-the-art methods on synthetic, natural image patch and the MNIST data sets.