mlsanityData to 14 Oct 2026

Image Clustering Conditioned on Text Criteria

arXiv 2310.18297 ↗First posted 27 Oct 2023cs.CV cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Classical clustering methods do not provide users with direct control of the clustering results, and the clustering results may not be consistent with the relevant criterion that a user has in mind. In this work, we present a new methodology for performing image clustering based on user-specified text criteria by leveraging modern vision-language models and large language models. We call our method Image Clustering Conditioned on Text Criteria (IC|TC), and it represents a different paradigm of image clustering. IC|TC requires a minimal and practical degree of human intervention and grants the user significant control over the clustering results in return. Our experiments show that IC|TC can effectively cluster images with various criteria, such as human action, physical location, or the person's mood, while significantly outperforming baselines.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 53 of its 84 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

53 of 84 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  4. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  5. 2021Grounded Language-Image Pre-trainingcited by 1,156
  6. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  8. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  9. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  10. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  11. 2020SCAN: Learning to Classify Images without Labelscited by 352
  12. 2022Visual Classification via Description from Large Language Modelscited by 336
  13. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
  14. 2016LIFT: Learned Invariant Feature Transformcited by 306
  15. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  16. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  17. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  18. 2016Large-Scale Image Retrieval with Attentive Deep Local Featurescited by 238
  19. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  20. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  21. 2015Particular object retrieval with integral max-pooling of CNN activationscited by 161
  22. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 148
  23. 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
  24. 2021SPICE: Semantic Pseudo-labeling for Image Clusteringcited by 104
  25. 2023Large Language Models Enable Few-Shot Clusteringcited by 72
  26. 2021Graph Contrastive Clusteringcited by 61
  27. 2022Twin Contrastive Learning for Online Clusteringcited by 60
  28. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  29. 2023From Sparse to Dense: GPT-4 Summarization with Chain of Density Promptingcited by 52
  30. 2020Improving Unsupervised Image Clustering With Robust Learningcited by 51
  31. 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attentioncited by 44
  32. 2018Local Descriptors Optimized for Average Precisioncited by 43
  33. 2021You Never Cluster Alonecited by 38
  34. 2019Local Features and Visual Words Emerge in Activationscited by 37
  35. 2023Image Clustering with External Guidancecited by 36
  36. 2022Semantic-Enhanced Image Clusteringcited by 25
  37. 2023DivClust: Controlling Diversity in Deep Clusteringcited by 18
  38. 2022Deep Fair Clustering via Maximizing and Minimizing Mutual Information: Theory, Algorithm and Metriccited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 10; 2026: 7.

26 papers.

Sort:

By area:

  1. 2024LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Modelcited by 44
  2. 2024Text Clustering as Classification with LLMscited by 16
  3. 2024Organizing Unstructured Image Collections using Natural Languagecited by 7
  4. 2024Generative Timelines for Instructed Visual Assemblycited by 5
  5. 2025Conditional Representation Learning for Customized Taskscited by 5
  6. 2025ConceptScope: Characterizing Dataset Bias via Disentangled Visual Conceptscited by 5
  7. 2024VLMine: Long-Tail Data Mining with Vision Language Modelscited by 4
  8. 2024Deep Spectral Clustering via Joint Spectral Embedding and Kmeanscited by 4
  9. 2025HERCULES: Hierarchical Embedding-based Recursive Clustering Using LLMs for Efficient Summarizationcited by 3
  10. 2025Open Ad-hoc Categorization with Contextualized Feature Learningcited by 3
  11. 2026The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metriccited by 3
  12. 2025Bootstrap Deep Spectral Clustering with Optimal Transportcited by 2
14 more
  1. 2024Text-Guided Alternative Image Clusteringcited by 1
  2. 2026CLAY: Conditional Visual Similarity Modulation in Vision-Language Embedding Spacecited by 1
  3. 2024Dual-Level Cross-Modal Contrastive Clusteringcited by 0
  4. 2024DSAI: Unbiased and Interpretable Latent Feature Extraction for Data-Centric AIcited by 0
  5. 2025Deep Clustering via Gradual Community Detectioncited by 0
  6. 2025Learning a Canonical Basis of Human Preferences from Binary Ratingscited by 0
  7. 2025Interpretable Text-Guided Image Clustering via Iterative Searchcited by 0
  8. 2025In-Context Clustering with Large Language Modelscited by 0
  9. 2025GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Taskscited by 0
  10. 2026Semantic Purification for Conditional Representation Learningcited by 0
  11. 2026Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centerscited by 0
  12. 2026Hierarchical Textual Knowledge for Enhanced Image Clusteringcited by 0
  13. 2026Universal Guideline-Driven Image Clustering via a Hybrid LLM Agentcited by 0
  14. 2026Controlling Embedding Spaces with Text-Conditioned Transformationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training
  2. 2024Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens
  3. 2024DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering
  4. 2024Agglomerative Token Clustering
  5. 2026Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering
  6. 2026Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Information
  7. 2025Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation
  8. 2025ESMC: MLLM-Based Embedding Selection for Explainable Multiple Clustering