# Walking the Tightrope: An Investigation of the Convolutional Autoencoder Bottleneck

arXiv: https://arxiv.org/abs/1911.07460 · HTML: https://mlsanity.com/paper/1911.07460/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [Skin feature point tracking using deep feature encodings](https://mlsanity.com/paper/2112.14159/) (2021)
- [Triggering Dark Showers with Conditional Dual Auto-Encoders](https://mlsanity.com/paper/2306.12955/) (2023)
- [Effects of Convolutional Autoencoder Bottleneck Width on StarGAN-based Singing Technique Conversion](https://mlsanity.com/paper/2308.10021/) (2023)

## Abstract (arXiv)

In this paper, we present an in-depth investigation of the convolutional autoencoder (CAE) bottleneck. Autoencoders (AE), and especially their convolutional variants, play a vital role in the current deep learning toolbox. Researchers and practitioners employ CAEs for a variety of tasks, ranging from outlier detection and compression to transfer and representation learning. Despite their widespread adoption, we have limited insight into how the bottleneck shape impacts the emergent properties of the CAE. We demonstrate that increased height and width of the bottleneck drastically improves generalization, which in turn leads to better performance of the latent codes in downstream transfer learning tasks. The number of channels in the bottleneck, on the other hand, is secondary in importance. Furthermore, we show empirically that, contrary to popular belief, CAEs do not learn to copy their input, even when the bottleneck has the same number of neurons as there are pixels in the input. Copying does not occur, despite training the CAE for 1,000 epochs on a tiny (≈ 600 images) dataset. We believe that the findings in this paper are directly applicable and will lead to improvements in models that rely on CAEs.