# Massively Parallel Reweighted Wake-Sleep

arXiv: https://arxiv.org/abs/2305.11022 · HTML: https://mlsanity.com/paper/2305.11022/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Reinterpreting Importance-Weighted Autoencoders](https://mlsanity.com/paper/1704.02916/) (2017)
- [Tensor Monte Carlo: particle methods for the GPU era](https://mlsanity.com/paper/1806.08593/) (2018)
- [Fighting Sample Degeneracy and Impoverishment in Particle Filters: A Review of Intelligent Approaches](https://mlsanity.com/paper/1308.2443/) (2013)
- [Divide-and-Conquer with Sequential Monte Carlo](https://mlsanity.com/paper/1406.4993/) (2014)
- [Variational Marginal Particle Filters](https://mlsanity.com/paper/2109.15134/) (2021)
- [Variational Inference with Locally Enhanced Bounds for Hierarchical Models](https://mlsanity.com/paper/2203.04432/) (2022)

## Built on by (most cited first, 1 of 1)

- [Massively Parallel Expectation Maximization For Approximate Posteriors](https://mlsanity.com/paper/2503.08264/) (2025)

## Abstract (arXiv)

Reweighted wake-sleep (RWS) is a machine learning method for performing Bayesian inference in a very general class of models. RWS draws K samples from an underlying approximate posterior, then uses importance weighting to provide a better estimate of the true posterior. RWS then updates its approximate posterior towards the importance-weighted estimate of the true posterior. However, recent work [Chattergee and Diaconis, 2018] indicates that the number of samples required for effective importance weighting is exponential in the number of latent variables. Attaining such a large number of importance samples is intractable in all but the smallest models. Here, we develop massively parallel RWS, which circumvents this issue by drawing K samples of all n latent variables, and individually reasoning about all Kn possible combinations of samples. While reasoning about Kn combinations might seem intractable, the required computations can be performed in polynomial time by exploiting conditional independencies in the generative model. We show considerable improvements over standard "global" RWS, which draws K samples from the full joint.