# Natural Evolution Strategies as a Black Box Estimator for Stochastic Variational Inference

arXiv: https://arxiv.org/abs/2308.08053 · HTML: https://mlsanity.com/paper/2308.08053/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [Transforming and Projecting Images into Class-conditional Generative Networks](https://mlsanity.com/paper/2005.01703/) (2020)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Learning to learn generative programs with Memoised Wake-Sleep](https://mlsanity.com/paper/2007.03132/) (2020)
- [Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods](https://mlsanity.com/paper/1906.03139/) (2019)
- [A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization](https://mlsanity.com/paper/1905.10474/) (2019)
- [Hybrid Memoised Wake-Sleep: Approximate Inference at the Discrete-Continuous Interface](https://mlsanity.com/paper/2107.06393/) (2021)

## Abstract (arXiv)

Stochastic variational inference and its derivatives in the form of variational autoencoders enjoy the ability to perform Bayesian inference on large datasets in an efficient manner. However, performing inference with a VAE requires a certain design choice (i.e. reparameterization trick) to allow unbiased and low variance gradient estimation, restricting the types of models that can be created. To overcome this challenge, an alternative estimator based on natural evolution strategies is proposed. This estimator does not make assumptions about the kind of distributions used, allowing for the creation of models that would otherwise not have been possible under the VAE framework.