# Image Clustering with External Guidance

arXiv: https://arxiv.org/abs/2310.11989 · HTML: https://mlsanity.com/paper/2310.11989/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2012.15409/) (2020)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Learning Discrete Representations via Information Maximizing Self-Augmented Training](https://mlsanity.com/paper/1702.08720/) (2017)
- [Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization](https://mlsanity.com/paper/1704.06327/) (2017)
- [SPICE: Semantic Pseudo-labeling for Image Clustering](https://mlsanity.com/paper/2103.09382/) (2021)
- [Deep Comprehensive Correlation Mining for Image Clustering](https://mlsanity.com/paper/1904.06925/) (2019)
- [Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation](https://mlsanity.com/paper/2106.00131/) (2021)
- [Graph Contrastive Clustering](https://mlsanity.com/paper/2104.01429/) (2021)
- [Learning Representation for Clustering via Prototype Scattering and Positive Sampling](https://mlsanity.com/paper/2111.11821/) (2021)
- [MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering](https://mlsanity.com/paper/2105.01899/) (2021)
- [You Never Cluster Alone](https://mlsanity.com/paper/2106.01908/) (2021)
- [Semantic-Enhanced Image Clustering](https://mlsanity.com/paper/2208.09849/) (2022)

## Built on by (most cited first, 30 of 36)

- [R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning](https://mlsanity.com/paper/2504.11195/) (2025)
- [Image Clustering Conditioned on Text Criteria](https://mlsanity.com/paper/2310.18297/) (2023)
- [A Survey on Deep Clustering: From the Prior Perspective](https://mlsanity.com/paper/2406.19602/) (2024)
- [Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2506.11036/) (2025)
- [On the Provable Importance of Gradients for Language-Assisted Image Clustering](https://mlsanity.com/paper/2510.16335/) (2025)
- [Organizing Unstructured Image Collections using Natural Language](https://mlsanity.com/paper/2410.05217/) (2024)
- [ZEUS: Zero-shot Embeddings for Unsupervised Separation of Tabular Data](https://mlsanity.com/paper/2505.10704/) (2025)
- [Transformers versus the EM Algorithm in Multi-class Clustering](https://mlsanity.com/paper/2502.06007/) (2025)
- [Towards Calibrated Deep Clustering Network](https://mlsanity.com/paper/2403.02998/) (2024)
- [DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering](https://mlsanity.com/paper/2412.18838/) (2024)
- [Open Ad-hoc Categorization with Contextualized Feature Learning](https://mlsanity.com/paper/2512.16202/) (2025)
- [SP2OT: Semantic-Regularized Progressive Partial Optimal Transport for Imbalanced Clustering](https://mlsanity.com/paper/2404.03446/) (2024)
- [Active Prompt Learning with Vision-Language Model Priors](https://mlsanity.com/paper/2411.16722/) (2024)
- [You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering](https://mlsanity.com/paper/2511.21193/) (2025)
- [Delving into Spectral Clustering with Vision-Language Representations](https://mlsanity.com/paper/2602.09586/) (2026)
- [Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representation](https://mlsanity.com/paper/2603.04803/) (2026)
- [Keep It Light! Simplifying Image Clustering Via Text-Free Adapters](https://mlsanity.com/paper/2502.04226/) (2025)
- [Crossmodal Knowledge Distillation with WordNet-Relaxed Text Embeddings for Robust Image Classification](https://mlsanity.com/paper/2503.24017/) (2025)
- [EDITS: Enhancing Dataset Distillation with Implicit Textual Semantics](https://mlsanity.com/paper/2509.13858/) (2025)
- [StarEmbed: Benchmarking Time Series Foundation Models on Astronomical Observations of Variable Stars](https://mlsanity.com/paper/2510.06200/) (2025)
- [Training-Free Test-Time Adaptation with Brownian Distance Covariance in Vision-Language Models](https://mlsanity.com/paper/2601.23253/) (2026)
- [CLUBench: A Clustering Benchmark](https://mlsanity.com/paper/2605.29933/) (2026)
- [Dual-Level Cross-Modal Contrastive Clustering](https://mlsanity.com/paper/2409.04561/) (2024)
- [Deep Clustering via Gradual Community Detection](https://mlsanity.com/paper/2501.02036/) (2025)
- [Interpretable Text-Guided Image Clustering via Iterative Search](https://mlsanity.com/paper/2506.12514/) (2025)
- [Semantic Purification for Conditional Representation Learning](https://mlsanity.com/paper/2602.05464/) (2026)
- [Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery](https://mlsanity.com/paper/2602.19910/) (2026)
- [Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers](https://mlsanity.com/paper/2603.24275/) (2026)
- [Hierarchical Textual Knowledge for Enhanced Image Clustering](https://mlsanity.com/paper/2604.11144/) (2026)
- [Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering](https://mlsanity.com/paper/2605.12961/) (2026)

## Abstract (arXiv)

The core of clustering is incorporating prior knowledge to construct supervision signals. From classic k-means based on data compactness to recent contrastive clustering guided by self-supervision, the evolution of clustering methods intrinsically corresponds to the progression of supervision signals. At present, substantial efforts have been devoted to mining internal supervision signals from data. Nevertheless, the abundant external knowledge such as semantic descriptions, which naturally conduces to clustering, is regrettably overlooked. In this work, we propose leveraging external knowledge as a new supervision signal to guide clustering, even though it seems irrelevant to the given data. To implement and validate our idea, we design an externally guided clustering method (Text-Aided Clustering, TAC), which leverages the textual semantics of WordNet to facilitate image clustering. Specifically, TAC first selects and retrieves WordNet nouns that best distinguish images to enhance the feature discriminability. Then, to improve image clustering performance, TAC collaborates text and image modalities by mutually distilling cross-modal neighborhood information. Experiments demonstrate that TAC achieves state-of-the-art performance on five widely used and three more challenging image clustering benchmarks, including the full ImageNet-1K dataset.