# Winner-Take-All Autoencoders

arXiv: https://arxiv.org/abs/1409.2752 · HTML: https://mlsanity.com/paper/1409.2752/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2014.

## Built on by (most cited first, 2 of 2)

- [Activation Learning by Local Competitions](https://mlsanity.com/paper/2209.13400/) (2022)
- [Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder](https://mlsanity.com/paper/2101.04053/) (2021)

## Abstract (arXiv)

In this paper, we propose a winner-take-all method for learning hierarchical sparse representations in an unsupervised fashion. We first introduce fully-connected winner-take-all autoencoders which use mini-batch statistics to directly enforce a lifetime sparsity in the activations of the hidden units. We then propose the convolutional winner-take-all autoencoder which combines the benefits of convolutional architectures and autoencoders for learning shift-invariant sparse representations. We describe a way to train convolutional autoencoders layer by layer, where in addition to lifetime sparsity, a spatial sparsity within each feature map is achieved using winner-take-all activation functions. We will show that winner-take-all autoencoders can be used to to learn deep sparse representations from the MNIST, CIFAR-10, ImageNet, Street View House Numbers and Toronto Face datasets, and achieve competitive classification performance.