# Semantic-Enhanced Image Clustering

arXiv: https://arxiv.org/abs/2208.09849 · HTML: https://mlsanity.com/paper/2208.09849/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 61 references that are in the map.
- First posted on arXiv 21 Aug 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training](https://mlsanity.com/paper/1908.06066/) (2019)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2012.15409/) (2020)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data](https://mlsanity.com/paper/2001.07966/) (2020)
- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [Deep Comprehensive Correlation Mining for Image Clustering](https://mlsanity.com/paper/1904.06925/) (2019)
- [Graph Contrastive Clustering](https://mlsanity.com/paper/2104.01429/) (2021)
- [Improving Unsupervised Image Clustering With Robust Learning](https://mlsanity.com/paper/2012.11150/) (2020)
- [Deep Discriminative Clustering Analysis](https://mlsanity.com/paper/1905.01681/) (2019)
- [Selective Pseudo-label Clustering](https://mlsanity.com/paper/2107.10692/) (2021)

## Built on by (most cited first, 25 of 25)

- [Image Clustering with External Guidance](https://mlsanity.com/paper/2310.11989/) (2023)
- [R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning](https://mlsanity.com/paper/2504.11195/) (2025)
- [Image Clustering Conditioned on Text Criteria](https://mlsanity.com/paper/2310.18297/) (2023)
- [ELFS: Label-Free Coreset Selection with Proxy Training Dynamics](https://mlsanity.com/paper/2406.04273/) (2024)
- [Hyperbolic Hierarchical Contrastive Hashing](https://mlsanity.com/paper/2212.08904/) (2022)
- [A Survey on Deep Clustering: From the Prior Perspective](https://mlsanity.com/paper/2406.19602/) (2024)
- [On the Provable Importance of Gradients for Language-Assisted Image Clustering](https://mlsanity.com/paper/2510.16335/) (2025)
- [LEMoN: Label Error Detection using Multimodal Neighbors](https://mlsanity.com/paper/2407.18941/) (2024)
- [ZEUS: Zero-shot Embeddings for Unsupervised Separation of Tabular Data](https://mlsanity.com/paper/2505.10704/) (2025)
- [Towards Calibrated Deep Clustering Network](https://mlsanity.com/paper/2403.02998/) (2024)
- [Bottleneck-Minimal Indexing for Generative Document Retrieval](https://mlsanity.com/paper/2405.10974/) (2024)
- [You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering](https://mlsanity.com/paper/2511.21193/) (2025)
- [Delving into Spectral Clustering with Vision-Language Representations](https://mlsanity.com/paper/2602.09586/) (2026)
- [Keep It Light! Simplifying Image Clustering Via Text-Free Adapters](https://mlsanity.com/paper/2502.04226/) (2025)
- [From Imitation to Innovation: The Emergence of AI Unique Artistic Styles and the Challenge of Copyright Protection](https://mlsanity.com/paper/2507.04769/) (2025)
- [Dual-Level Cross-Modal Contrastive Clustering](https://mlsanity.com/paper/2409.04561/) (2024)
- [Deep Clustering via Gradual Community Detection](https://mlsanity.com/paper/2501.02036/) (2025)
- [Improve Contrastive Clustering Performance by Multiple Fusing-Augmenting ViT Blocks](https://mlsanity.com/paper/2511.08883/) (2025)
- [Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers](https://mlsanity.com/paper/2603.24275/) (2026)
- [Hierarchical Textual Knowledge for Enhanced Image Clustering](https://mlsanity.com/paper/2604.11144/) (2026)
- [Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering](https://mlsanity.com/paper/2605.12961/) (2026)
- [Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling](https://mlsanity.com/paper/2605.23198/) (2026)
- [Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Information](https://mlsanity.com/paper/2608.08418/) (2026)
- [CAESAR: Clustering via Autonomous Embedding-Space Agglomerative Reorganization](https://mlsanity.com/paper/2609.32570/) (2026)
- [VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training](https://mlsanity.com/paper/2609.36648/) (2026)

## Abstract (arXiv)

Image clustering is an important and open-challenging task in computer vision. Although many methods have been proposed to solve the image clustering task, they only explore images and uncover clusters according to the image features, thus being unable to distinguish visually similar but semantically different images. In this paper, we propose to investigate the task of image clustering with the help of a visual-language pre-training model. Different from the zero-shot setting, in which the class names are known, we only know the number of clusters in this setting. Therefore, how to map images to a proper semantic space and how to cluster images from both image and semantic spaces are two key problems. To solve the above problems, we propose a novel image clustering method guided by the visual-language pre-training model CLIP, named Semantic-Enhanced Image Clustering (SIC). In this new method, we propose a method to map the given images to a proper semantic space first and efficient methods to generate pseudo-labels according to the relationships between images and semantics. Finally, we propose performing clustering with consistency learning in both image space and semantic space, in a self-supervised learning fashion. The theoretical result of convergence analysis shows that our proposed method can converge at a sublinear speed. Theoretical analysis of expectation risk also shows that we can reduce the expected risk by improving neighborhood consistency, increasing prediction confidence, or reducing neighborhood imbalance. Experimental results on five benchmark datasets clearly show the superiority of our new method.