# Stacked What-Where Auto-encoders

arXiv: https://arxiv.org/abs/1506.02351 · HTML: https://mlsanity.com/paper/1506.02351/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2015.

## Built on by (most cited first, 20 of 20)

- [MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering](https://mlsanity.com/paper/2105.01899/) (2021)
- [Unsupervised Hyperbolic Representation Learning via Message Passing Auto-Encoders](https://mlsanity.com/paper/2103.16046/) (2021)
- [A Comparative Review of Recent Few-Shot Object Detection Algorithms](https://mlsanity.com/paper/2111.00201/) (2021)
- [Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition](https://mlsanity.com/paper/2110.09018/) (2021)
- [Improving memory banks for unsupervised learning with large mini-batch, consistency and hard negative mining](https://mlsanity.com/paper/2102.04442/) (2021)
- [DECK: Model Hardening for Defending Pervasive Backdoors](https://mlsanity.com/paper/2206.09272/) (2022)
- [Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations](https://mlsanity.com/paper/2101.06848/) (2021)
- [Neural Mixture Models with Expectation-Maximization for End-to-end Deep Clustering](https://mlsanity.com/paper/2107.02453/) (2021)
- [ReRankMatch: Semi-Supervised Learning with Semantics-Oriented Similarity Representation](https://mlsanity.com/paper/2102.06328/) (2021)
- [Learning the Precise Feature for Cluster Assignment](https://mlsanity.com/paper/2106.06159/) (2021)
- [WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification](https://mlsanity.com/paper/2308.14995/) (2023)
- [Biologically Inspired Hexagonal Deep Learning for Hexagonal Image Generation](https://mlsanity.com/paper/2101.00337/) (2021)
- [Concurrent Discrimination and Alignment for Self-Supervised Feature Learning](https://mlsanity.com/paper/2108.08562/) (2021)
- [RankingMatch: Delving into Semi-Supervised Learning with Consistency Regularization and Ranking Loss](https://mlsanity.com/paper/2110.04430/) (2021)
- [ExpertNet: A Symbiosis of Classification and Clustering](https://mlsanity.com/paper/2201.06344/) (2022)
- [Semi-Supervised Super-Resolution](https://mlsanity.com/paper/2204.08192/) (2022)
- ['Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation](https://mlsanity.com/paper/2208.01840/) (2022)
- [Contrastive Representation Disentanglement for Clustering](https://mlsanity.com/paper/2306.05439/) (2023)
- [An Adaptor for Triggering Semi-Supervised Learning to Out-of-Box Serve Deep Image Clustering](https://mlsanity.com/paper/2509.20976/) (2025)
- [What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localization](https://mlsanity.com/paper/2605.12021/) (2026)

## Abstract (arXiv)

We present a novel architecture, the "stacked what-where auto-encoders" (SWWAE), which integrates discriminative and generative pathways and provides a unified approach to supervised, semi-supervised and unsupervised learning without relying on sampling during training. An instantiation of SWWAE uses a convolutional net (Convnet) (LeCun et al. (1998)) to encode the input, and employs a deconvolutional net (Deconvnet) (Zeiler et al. (2010)) to produce the reconstruction. The objective function includes reconstruction terms that induce the hidden states in the Deconvnet to be similar to those of the Convnet. Each pooling layer produces two sets of variables: the "what" which are fed to the next layer, and its complementary variable "where" that are fed to the corresponding layer in the generative decoder.