mlsanityData to 14 Oct 2026

Exploring the Limits of Deep Image Clustering using Pretrained Models

arXiv 2303.17896 ↗First posted 31 Mar 2023cs.CV cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

We present a general methodology that learns to classify images without labels by leveraging pretrained feature extractors. Our approach involves self-distillation training of clustering heads based on the fact that nearest neighbours in the pretrained feature space are likely to share the same label. We propose a novel objective that learns associations between image features by introducing a variant of pointwise mutual information together with instance weighting. We demonstrate that the proposed objective is able to attenuate the effect of false positive pairs while efficiently exploiting the structure in the pretrained feature space. As a result, we improve the clustering accuracy over k-means on 17 different pretrained models by 6.1% and 12.2% on ImageNet and CIFAR100, respectively. Finally, using self-supervised vision transformers, we achieve a clustering accuracy of 61.6% on ImageNet. The code is available at https://github.com/HHU-MMBS/TEMI-official-BMVC2023.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 31 Mar 2023.

Builds on

37 of 51 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
25 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  2. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  4. 2017Decoupled Weight Decay Regularizationcited by 1,831
  5. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  8. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  9. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  10. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  11. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  12. 2021Intriguing Properties of Vision Transformerscited by 446
  13. 2020SCAN: Learning to Classify Images without Labelscited by 352
  14. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  15. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  16. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 256
  17. 2016Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clusteringcited by 237
  18. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  19. 2021SPICE: Semantic Pseudo-labeling for Image Clusteringcited by 104
  20. 2019Deep Comprehensive Correlation Mining for Image Clusteringcited by 80
  21. 2019Unsupervised Deep Learning by Neighbourhood Discoverycited by 78
  22. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  23. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  24. 2021Self-Supervised Classification Networkcited by 28
  25. 2022Deep Clustering with Features from Self-Supervised Pretrainingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 12; 2025: 13; 2026: 5.

33 papers.

Sort:

By area:

  1. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  2. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  3. 2023Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Modelscited by 22
  4. 2024MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representationscited by 19
  5. 2025I-Con: A Unifying Framework for Representation Learningcited by 16
  6. 2024ELFS: Label-Free Coreset Selection with Proxy Training Dynamicscited by 10
  7. 2025Exploring a Principled Framework for Deep Subspace Clusteringcited by 7
  8. 2023CEIR: Concept-based Explainable Image Representation Learningcited by 5
  9. 2024Rethinking cluster-conditioned diffusion models for label-free image synthesiscited by 5
  10. 2024Style-based Clustering of Visual Artworks and the Play of Neural Style-Representationscited by 3
  11. 2024From Logits to Hierarchies: Hierarchical Clustering made Simplecited by 3
  12. 2024TreeDiffusion: Hierarchical Generative Clustering for Conditional Diffusioncited by 3
21 more
  1. 2025Clustering via Self-Supervised Diffusioncited by 3
  2. 2023pNNCLR: Stochastic Pseudo Neighborhoods for Contrastive Learning based Unsupervised Representation Learning Problemscited by 2
  3. 2024Deep Online Probability Aggregation Clusteringcited by 2
  4. 2024Improving Image Clustering with Artifacts Attenuation via Inference-Time Attention Engineeringcited by 2
  5. 2024Scaling Up Deep Clustering Methods Beyond ImageNet-1Kcited by 1
  6. 2024Text-Guided Alternative Image Clusteringcited by 1
  7. 2025Keep It Light! Simplifying Image Clustering Via Text-Free Adapterscited by 1
  8. 2025Self-Enhanced Image Clustering with Cross-Modal Semantic Consistencycited by 1
  9. 2025Beyond I-Con: Exploring New Dimension of Distance Measures in Representation Learningcited by 1
  10. 2024Deep Clustering of Remote Sensing Scenes through Heterogeneous Transfer Learningcited by 0
  11. 2025ConstellationNet: Reinventing Spatial Clustering through GNNscited by 0
  12. 2025Utilization of Neighbor Information for Image Classification with Different Levels of Supervisioncited by 0
  13. 2025Dimensionally Reduced Open-World Clustering: DROWCULAcited by 0
  14. 2025An Adaptor for Triggering Semi-Supervised Learning to Out-of-Box Serve Deep Image Clusteringcited by 0
  15. 2025ClusterMine: Robust Label-Free Visual Out-Of-Distribution Detection via Concept Mining from Text Corporacited by 0
  16. 2025Unsupervised Image Classification with Adaptive Nearest Neighbor Selection and Cluster Ensemblescited by 0
  17. 2026Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centerscited by 0
  18. 2026Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labelingcited by 0
  19. 2026Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Informationcited by 0
  20. 2026InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomycited by 0
  21. 2026VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Domain-Agnostic Clustering with Self-Distillation
  2. 2024Deep Clustering with Diffused Sampling and Hardness-aware Self-distillation
  3. 2025Object-level Self-Distillation for Vision Pretraining
  4. 2025Dataset Distillation for Pre-Trained Self-Supervised Vision Models
  5. 2026Hierarchical Textual Knowledge for Enhanced Image Clustering
  6. 2025Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks
  7. 2024DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering
  8. 2024Scaling Backwards: Minimal Synthetic Pre-training?