# Stochastic Backpropagation and Approximate Inference in Deep Generative Models

arXiv: https://arxiv.org/abs/1401.4082 · HTML: https://mlsanity.com/paper/1401.4082/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2014.

## Built on by (most cited first, 30 of 31)

- [DetCo: Unsupervised Contrastive Learning for Object Detection](https://mlsanity.com/paper/2102.04803/) (2021)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [Compressive Visual Representations](https://mlsanity.com/paper/2109.12909/) (2021)
- [Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2110.05340/) (2021)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [Unsupervised Visual Representation Learning by Synchronous Momentum Grouping](https://mlsanity.com/paper/2207.06167/) (2022)
- [Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning](https://mlsanity.com/paper/2303.17142/) (2023)
- [DEFT: Dexterous Fine-Tuning for Real-World Hand Policies](https://mlsanity.com/paper/2310.19797/) (2023)
- [VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models](https://mlsanity.com/paper/2601.14354/) (2026)
- [Entropic Issues in Likelihood-Based OOD Detection](https://mlsanity.com/paper/2109.10794/) (2021)
- [Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations](https://mlsanity.com/paper/2202.10951/) (2022)
- [Object-Centric Learning with Slot Mixture Module](https://mlsanity.com/paper/2311.04640/) (2023)
- [Generalization Gap in Amortized Inference](https://mlsanity.com/paper/2205.11640/) (2022)
- [TopicNet: Semantic Graph-Guided Topic Discovery](https://mlsanity.com/paper/2110.14286/) (2021)
- [Vitruvio: 3D Building Meshes via Single Perspective Sketches](https://mlsanity.com/paper/2210.13634/) (2022)
- [Learning Invariances with Generalised Input-Convex Neural Networks](https://mlsanity.com/paper/2204.07009/) (2022)
- [CHIMLE: Conditional Hierarchical IMLE for Multimodal Conditional Image Synthesis](https://mlsanity.com/paper/2211.14286/) (2022)
- [In-Context Parametric Inference: Point or Distribution Estimators?](https://mlsanity.com/paper/2502.11617/) (2025)
- [Leveraging Structured Biological Knowledge for Counterfactual Inference: a Case Study of Viral Pathogenesis](https://mlsanity.com/paper/2101.05136/) (2021)
- [The use of Generative Adversarial Networks to characterise new physics in multi-lepton final states at the LHC](https://mlsanity.com/paper/2105.14933/) (2021)
- [toon2real: Translating Cartoon Images to Realistic Images](https://mlsanity.com/paper/2102.01143/) (2021)
- [Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task](https://mlsanity.com/paper/2106.00537/) (2021)
- [Learning to sample in Cartesian MRI](https://mlsanity.com/paper/2312.04327/) (2023)
- [A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models](https://mlsanity.com/paper/2605.06829/) (2026)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [Topographic VAEs learn Equivariant Capsules](https://mlsanity.com/paper/2109.01394/) (2021)
- [Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods](https://mlsanity.com/paper/2308.00129/) (2023)
- [Unsupervised Text Style Transfer with Deep Generative Models](https://mlsanity.com/paper/2308.16584/) (2023)
- [The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need](https://mlsanity.com/paper/2312.05695/) (2023)
- [Sample-Smooth Spaces: A Convenient Category for Differentiable Probabilistic Programming](https://mlsanity.com/paper/2609.26270/) (2026)

## Abstract (arXiv)

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.