Improved Auto-Encoding using Deterministic Projected Belief Networks
Not on the map. Why?
Abstract (arXiv)
In this paper, we exploit the unique properties of a deterministic projected belief network (D-PBN) to take full advantage of trainable compound activation functions (TCAs). A D-PBN is a type of auto-encoder that operates by "backing up" through a feed-forward neural network. TCAs are activation functions with complex monotonic-increasing shapes that change the distribution of the data so that the linear transformation that follows is more effective. Because a D-PBN operates by "backing up", the TCAs are inverted in the reconstruction process, restoring the original distribution of the data, thus taking advantage of a given TCA in both analysis and reconstruction. In this paper, we show that a D-PBN auto-encoder with TCAs can significantly out-perform standard auto-encoders including variational auto-encoders.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 23 references that are in the map. First posted on arXiv 14 Sep 2023.
Builds on
9 of 23 references.
References · 9
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
- 2019Neural Spline Flowscited by 590
- 2020SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flowscited by 71
- 2019Pad'e Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networkscited by 65
- 2019Cubic-Spline Flowscited by 34
- 2020A Neural Network Based on First Principlescited by 3
- 2022Trainable Compound Activation Functions for Machine Learningcited by 2
- 2022Using the Projected Belief Network at High Dimensionscited by 2
Built on by
Citing papers per year: 2024: 2.
2 papers.