# Unsupervised Learning of Structured Representations via Closed-Loop Transcription

arXiv: https://arxiv.org/abs/2210.16782 · HTML: https://mlsanity.com/paper/2210.16782/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 30 Oct 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Context Autoencoder for Self-Supervised Representation Learning](https://mlsanity.com/paper/2202.03026/) (2022)
- [Data Augmentation Generative Adversarial Networks](https://mlsanity.com/paper/1711.04340/) (2017)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction](https://mlsanity.com/paper/2006.08558/) (2020)
- [Self-Supervised GANs via Auxiliary Rotation Loss](https://mlsanity.com/paper/1811.11212/) (2018)
- [ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction](https://mlsanity.com/paper/2105.10446/) (2021)
- [ClusterGAN : Latent Space Clustering in Generative Adversarial Networks](https://mlsanity.com/paper/1809.03627/) (2018)
- [Dual Contradistinctive Generative Autoencoder](https://mlsanity.com/paper/2011.10063/) (2020)
- [Diverse Image Generation via Self-Conditioned GANs](https://mlsanity.com/paper/2006.10728/) (2020)
- [Training GANs with Stronger Augmentations via Contrastive Discriminator](https://mlsanity.com/paper/2103.09742/) (2021)
- [Neural Manifold Clustering and Embedding](https://mlsanity.com/paper/2201.10000/) (2022)
- [Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction](https://mlsanity.com/paper/2111.06636/) (2021)
- [Incremental Learning of Structured Memory via Closed-Loop Transcription](https://mlsanity.com/paper/2202.05411/) (2022)
- [Variational Clustering: Leveraging Variational Autoencoders for Image Clustering](https://mlsanity.com/paper/2005.04613/) (2020)

## Built on by (most cited first, 7 of 7)

- [Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models](https://mlsanity.com/paper/2306.05272/) (2023)
- [Simplifying DINO via Coding Rate Regularization](https://mlsanity.com/paper/2502.10385/) (2025)
- [Top-Down Visual Attention from Analysis by Synthesis](https://mlsanity.com/paper/2303.13043/) (2023)
- [EMP-SSL: Towards Self-Supervised Learning in One Training Epoch](https://mlsanity.com/paper/2304.03977/) (2023)
- [Closed-Loop Transcription via Convolutional Sparse Coding](https://mlsanity.com/paper/2302.09347/) (2023)
- [Unifying Self-Supervised Clustering and Energy-Based Models](https://mlsanity.com/paper/2401.00873/) (2023)
- [On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation](https://mlsanity.com/paper/2609.21001/) (2026)

## Abstract (arXiv)

This paper proposes an unsupervised method for learning a unified representation that serves both discriminative and generative purposes. While most existing unsupervised learning approaches focus on a representation for only one of these two goals, we show that a unified representation can enjoy the mutual benefits of having both. Such a representation is attainable by generalizing the recently proposed textitclosed-loop transcription framework, known as CTRL, to the unsupervised setting. This entails solving a constrained maximin game over a rate reduction objective that expands features of all samples while compressing features of augmentations of each sample. Through this process, we see discriminative low-dimensional structures emerge in the resulting representations. Under comparable experimental conditions and network complexities, we demonstrate that these structured representations enable classification performance close to state-of-the-art unsupervised discriminative representations, and conditionally generated image quality significantly higher than that of state-of-the-art unsupervised generative models. Source code can be found at https://github.com/Delay-Xili/uCTRL.