# Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation

arXiv: https://arxiv.org/abs/2106.00131 · HTML: https://mlsanity.com/paper/2106.00131/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [Learning Discrete Representations via Information Maximizing Self-Augmented Training](https://mlsanity.com/paper/1702.08720/) (2017)
- [Unsupervised Learning by Predicting Noise](https://mlsanity.com/paper/1704.05310/) (2017)
- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [Deep Comprehensive Correlation Mining for Image Clustering](https://mlsanity.com/paper/1904.06925/) (2019)
- [RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets](https://mlsanity.com/paper/1812.02293/) (2018)

## Built on by (most cited first, 30 of 65)

- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions](https://mlsanity.com/paper/2206.07579/) (2022)
- [ClusterLLM: Large Language Models as a Guide for Text Clustering](https://mlsanity.com/paper/2305.14871/) (2023)
- [Visual Recognition with Deep Nearest Centroids](https://mlsanity.com/paper/2209.07383/) (2022)
- [Learning Representation for Clustering via Prototype Scattering and Positive Sampling](https://mlsanity.com/paper/2111.11821/) (2021)
- [Neural Manifold Clustering and Embedding](https://mlsanity.com/paper/2201.10000/) (2022)
- [You Never Cluster Alone](https://mlsanity.com/paper/2106.01908/) (2021)
- [Image Clustering with External Guidance](https://mlsanity.com/paper/2310.11989/) (2023)
- [Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models](https://mlsanity.com/paper/2306.05272/) (2023)
- [DivClust: Controlling Diversity in Deep Clustering](https://mlsanity.com/paper/2304.01042/) (2023)
- [VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distribution](https://mlsanity.com/paper/2304.01434/) (2023)
- [Text Clustering as Classification with LLMs](https://mlsanity.com/paper/2410.00927/) (2024)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Adversarial Contrastive Learning via Asymmetric InfoNCE](https://mlsanity.com/paper/2207.08374/) (2022)
- [P2OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering](https://mlsanity.com/paper/2401.09266/) (2024)
- [Contextually Affinitive Neighborhood Refinery for Deep Clustering](https://mlsanity.com/paper/2312.07806/) (2023)
- [Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Mask](https://mlsanity.com/paper/2405.05959/) (2024)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [On the Provable Importance of Gradients for Language-Assisted Image Clustering](https://mlsanity.com/paper/2510.16335/) (2025)
- [Federated Momentum Contrastive Clustering](https://mlsanity.com/paper/2206.05093/) (2022)
- [Learning Weakly-Supervised Contrastive Representations](https://mlsanity.com/paper/2202.06670/) (2022)
- [Paying More Attention to Self-attention: Improving Pre-trained Language Models via Attention Guiding](https://mlsanity.com/paper/2204.02922/) (2022)
- [Contextuality Helps Representation Learning for Generalized Category Discovery](https://mlsanity.com/paper/2407.19752/) (2024)
- [Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification](https://mlsanity.com/paper/2412.16780/) (2024)
- [Generalised Mutual Information for Discriminative Clustering](https://mlsanity.com/paper/2210.06300/) (2022)
- [Hierarchical Semantic Alignment for Image Clustering](https://mlsanity.com/paper/2512.00904/) (2025)
- [Personalized Clustering via Targeted Representation Learning](https://mlsanity.com/paper/2412.13690/) (2024)
- [Cluster Analysis with Deep Embeddings and Contrastive Learning](https://mlsanity.com/paper/2109.12714/) (2021)
- [Few-Shot Class-Incremental Learning with Prior Knowledge](https://mlsanity.com/paper/2402.01201/) (2024)
- [CCFC++: Enhancing Federated Clustering through Feature Decorrelation](https://mlsanity.com/paper/2402.12852/) (2024)

## Abstract (arXiv)

Clustering is one of the most fundamental tasks in machine learning. Recently, deep clustering has become a major trend in clustering techniques. Representation learning often plays an important role in the effectiveness of deep clustering, and thus can be a principal cause of performance degradation. In this paper, we propose a clustering-friendly representation learning method using instance discrimination and feature decorrelation. Our deep-learning-based representation learning method is motivated by the properties of classical spectral clustering. Instance discrimination learns similarities among data and feature decorrelation removes redundant correlation among features. We utilize an instance discrimination method in which learning individual instance classes leads to learning similarity among instances. Through detailed experiments and examination, we show that the approach can be adapted to learning a latent space for clustering. We design novel softmax-formulated decorrelation constraints for learning. In evaluations of image clustering using CIFAR-10 and ImageNet-10, our method achieves accuracy of 81.5% and 95.4%, respectively. We also show that the softmax-formulated constraints are compatible with various neural networks.