# Improved Auto-Encoding using Deterministic Projected Belief Networks

arXiv: https://arxiv.org/abs/2309.07481 · HTML: https://mlsanity.com/paper/2309.07481/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows](https://mlsanity.com/paper/2007.02731/) (2020)
- [Pad'e Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks](https://mlsanity.com/paper/1907.06732/) (2019)
- [Cubic-Spline Flows](https://mlsanity.com/paper/1906.02145/) (2019)
- [A Neural Network Based on First Principles](https://mlsanity.com/paper/2002.07469/) (2020)
- [Trainable Compound Activation Functions for Machine Learning](https://mlsanity.com/paper/2204.12920/) (2022)
- [Using the Projected Belief Network at High Dimensions](https://mlsanity.com/paper/2204.12922/) (2022)

## Built on by (most cited first, 2 of 2)

- [Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions](https://mlsanity.com/paper/2402.09249/) (2024)
- [Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs](https://mlsanity.com/paper/2401.11199/) (2024)

## Abstract (arXiv)

In this paper, we exploit the unique properties of a deterministic projected belief network (D-PBN) to take full advantage of trainable compound activation functions (TCAs). A D-PBN is a type of auto-encoder that operates by "backing up" through a feed-forward neural network. TCAs are activation functions with complex monotonic-increasing shapes that change the distribution of the data so that the linear transformation that follows is more effective. Because a D-PBN operates by "backing up", the TCAs are inverted in the reconstruction process, restoring the original distribution of the data, thus taking advantage of a given TCA in both analysis and reconstruction. In this paper, we show that a D-PBN auto-encoder with TCAs can significantly out-perform standard auto-encoders including variational auto-encoders.