# Deep Clustering by Semantic Contrastive Learning

arXiv: https://arxiv.org/abs/2103.02662 · HTML: https://mlsanity.com/paper/2103.02662/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Local Aggregation for Unsupervised Learning of Visual Embeddings](https://mlsanity.com/paper/1903.12355/) (2019)
- [Online Deep Clustering for Unsupervised Representation Learning](https://mlsanity.com/paper/2006.10645/) (2020)
- [Deep Subspace Clustering Networks](https://mlsanity.com/paper/1709.02508/) (2017)
- [Deep Comprehensive Correlation Mining for Image Clustering](https://mlsanity.com/paper/1904.06925/) (2019)
- [Unsupervised Deep Learning by Neighbourhood Discovery](https://mlsanity.com/paper/1904.11567/) (2019)
- [GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering](https://mlsanity.com/paper/2002.11863/) (2020)
- [Deep Discriminative Clustering Analysis](https://mlsanity.com/paper/1905.01681/) (2019)
- [Learning Markov Clustering Networks for Scene Text Detection](https://mlsanity.com/paper/1805.08365/) (2018)
- [Deep Image Clustering with Category-Style Representation](https://mlsanity.com/paper/2007.10004/) (2020)

## Built on by (most cited first, 11 of 11)

- [Dynamic Conceptional Contrastive Learning for Generalized Category Discovery](https://mlsanity.com/paper/2303.17393/) (2023)
- [A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions](https://mlsanity.com/paper/2206.07579/) (2022)
- [You Never Cluster Alone](https://mlsanity.com/paper/2106.01908/) (2021)
- [Dense Semantic Contrast for Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2109.07756/) (2021)
- [Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning](https://mlsanity.com/paper/2202.08480/) (2022)
- [DivClust: Controlling Diversity in Deep Clustering](https://mlsanity.com/paper/2304.01042/) (2023)
- [Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentation](https://mlsanity.com/paper/2303.16521/) (2023)
- [Beyond Known Clusters: Probe New Prototypes for Efficient Generalized Class Discovery](https://mlsanity.com/paper/2404.08995/) (2024)
- [Disentangling Learnable and Memorizable Data via Contrastive Learning for Semantic Communications](https://mlsanity.com/paper/2212.09071/) (2022)
- [INTENT: Trajectory Prediction Framework with Intention-Guided Contrastive Clustering](https://mlsanity.com/paper/2503.04952/) (2025)
- [Generalized Fine-Grained Category Discovery with Multi-Granularity Conceptual Experts](https://mlsanity.com/paper/2509.26227/) (2025)

## Abstract (arXiv)

Whilst contrastive learning has recently brought notable benefits to deep clustering of unlabelled images by learning sample-specific discriminative visual features, its potential for explicitly inferring class decision boundaries is less well understood. This is because its instance discrimination strategy is not class sensitive, therefore, the clusters derived on the resulting sample-specific feature space are not optimised for corresponding to meaningful class decision boundaries. In this work, we solve this problem by introducing Semantic Contrastive Learning (SCL). SCL imposes explicitly distance-based cluster structures on unlabelled training data by formulating a semantic (cluster-aware) contrastive learning objective. Moreover, we introduce a clustering consistency condition to be satisfied jointly by both instance visual similarities and cluster decision boundaries, and concurrently optimising both to reason about the hypotheses of semantic ground-truth classes (unknown/unlabelled) on-the-fly by their consensus. This semantic contrastive learning approach to discovering unknown class decision boundaries has considerable advantages to unsupervised learning of object recognition tasks. Extensive experiments show that SCL outperforms state-of-the-art contrastive learning and deep clustering methods on six object recognition benchmarks, especially on the more challenging finer-grained and larger datasets.