# PatchVAE: Learning Local Latent Codes for Recognition

arXiv: https://arxiv.org/abs/2004.03623 · HTML: https://mlsanity.com/paper/2004.03623/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Diffusion Models Beat GANs on Image Classification](https://mlsanity.com/paper/2307.08702/) (2023)
- [Do text-free diffusion models learn discriminative visual representations?](https://mlsanity.com/paper/2311.17921/) (2023)
- [GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation](https://mlsanity.com/paper/2112.01036/) (2021)
- [Unsupervised Causal Binary Concepts Discovery with VAE for Black-box Model Explanation](https://mlsanity.com/paper/2109.04518/) (2021)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Unsupervised Learning of Structured Representations via Closed-Loop Transcription](https://mlsanity.com/paper/2210.16782/) (2022)
- [PatchGame: Learning to Signal Mid-level Patches in Referential Games](https://mlsanity.com/paper/2111.01785/) (2021)
- [One-shot Ultra-high-Resolution Generative Adversarial Network That Synthesizes 16K Images On A Single GPU](https://mlsanity.com/paper/2202.13799/) (2022)
- [Deep Contrastive Patch-Based Subspace Learning for Camera Image Signal Processing](https://mlsanity.com/paper/2104.00253/) (2021)
- [Multibranch Generative Models for Multichannel Imaging with an Application to PET/CT Synergistic Reconstruction](https://mlsanity.com/paper/2404.08748/) (2024)

## Abstract (arXiv)

Unsupervised representation learning holds the promise of exploiting large amounts of unlabeled data to learn general representations. A promising technique for unsupervised learning is the framework of Variational Auto-encoders (VAEs). However, unsupervised representations learned by VAEs are significantly outperformed by those learned by supervised learning for recognition. Our hypothesis is that to learn useful representations for recognition the model needs to be encouraged to learn about repeating and consistent patterns in data. Drawing inspiration from the mid-level representation discovery work, we propose PatchVAE, that reasons about images at patch level. Our key contribution is a bottleneck formulation that encourages mid-level style representations in the VAE framework. Our experiments demonstrate that representations learned by our method perform much better on the recognition tasks compared to those learned by vanilla VAEs.