# Maximum Entropy Auto-Encoding

arXiv: https://arxiv.org/abs/2104.07448 · HTML: https://mlsanity.com/paper/2104.07448/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 21 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 4 of 4)

- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function](https://mlsanity.com/paper/1907.08956/) (2019)
- [A Neural Network Based on First Principles](https://mlsanity.com/paper/2002.07469/) (2020)
- [The Projected Belief Network Classfier : both Generative and Discriminative](https://mlsanity.com/paper/2008.06434/) (2020)

## Abstract (arXiv)

In this paper, it is shown that an auto-encoder using optimal reconstruction significantly outperforms a conventional auto-encoder. Optimal reconstruction uses the conditional mean of the input given the features, under a maximum entropy prior distribution. The optimal reconstruction network, which is called deterministic projected belied network (D-PBN), resembles a standard reconstruction network, but with special non-linearities that mist be iteratively solved. The method, which can be seen as a generalization of maximum entropy image reconstruction, extends to multiple layers. In experiments, mean square reconstruction error reduced by up to a factor of two. The performance improvement diminishes for deeper networks, or for input data with unconstrained values (Gaussian assumption).