mlsanityData to 14 Oct 2026

Universal Guideline-Driven Image Clustering via a Hybrid LLM Agent

arXiv 2606.24094 ↗First posted 23 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Unifying image clustering across different clustering scenarios remains challenging due to fundamental gaps among tasks. We introduce a Guideline-Driven Image Clustering Agent, the first universal framework that bridges these gaps through textual guidelines. To incorporate complex guidelines without task-specific training, we propose Generative Concept Proxy Modeling, which generates guideline-aware embeddings via concept proxy extraction. For scenarios requiring automatic cluster discovery, we introduce LLM Traversal based on Minimum Spanning Tree that selectively applies LLM reasoning for complex semantic judgments. Our method generalizes across diverse clustering scenarios spanning from general to fine-grained categorization, from global to local criteria, and from balanced to long-tail distributions. Our framework consistently outperforms specialized methods across diverse clustering tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 60 of its 94 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

60 of 94 references.

References · 60

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  9. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  10. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  11. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  12. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
38 more
  1. 2020SCAN: Learning to Classify Images without Labelscited by 352
  2. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  3. 2023Improving Text Embeddings with Large Language Modelscited by 303
  4. 2020Contrastive Clusteringcited by 265
  5. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  6. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  7. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  8. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  9. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 126
  10. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  11. 2021SPICE: Semantic Pseudo-labeling for Image Clusteringcited by 104
  12. 2019Deep Comprehensive Correlation Mining for Image Clusteringcited by 80
  13. 2023Large Language Models Enable Few-Shot Clusteringcited by 72
  14. 2021Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelationcited by 65
  15. 2021Graph Contrastive Clusteringcited by 61
  16. 2022Twin Contrastive Learning for Online Clusteringcited by 60
  17. 2018FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discoverycited by 58
  18. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  19. 2024LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Modelcited by 44
  20. 2021MiCE: Mixture of Contrastive Experts for Unsupervised Image Clusteringcited by 42
  21. 2021You Never Cluster Alonecited by 38
  22. 2023Image Clustering with External Guidancecited by 36
  23. 2019MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generationcited by 29
  24. 2019OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clusteringcited by 28
  25. 2023Image Clustering Conditioned on Text Criteriacited by 26
  26. 2023Stable Cluster Discrimination for Deep Clusteringcited by 24
  27. 2024Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentationcited by 24
  28. 2024Multi-Modal Proxy Learning Towards Personalized Visual Multiple Clusteringcited by 21
  29. 2023DivClust: Controlling Diversity in Deep Clusteringcited by 18
  30. 2023AugDMC: Data Augmentation Guided Deep Multiple Clusteringcited by 15
  31. 2018Improving Image Clustering With Multiple Pretrained CNN Feature Extractorscited by 12
  32. 2024Customized Multiple Clustering via Multi-Modal Subspace Proxy Learningcited by 12
  33. 2025Forensic Self-Descriptions Are All You Need for Zero-Shot Detection, Open-Set Source Attribution, and Clustering of AI-generated Imagescited by 11
  34. 2023Contextually Affinitive Neighborhood Refinery for Deep Clusteringcited by 10
  35. 2021Contrastive Fine-grained Class Clustering via Generative Adversarial Networkscited by 9
  36. 2024Control-oriented Clustering of Visual Latent Representationcited by 8
  37. 2024Compositional Image Retrieval via Instruction-Aware Contrastive Learningcited by 8
  38. 2024Organizing Unstructured Image Collections using Natural Languagecited by 7

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hierarchical Textual Knowledge for Enhanced Image Clustering
  2. 2026Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering
  3. 2026Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers
  4. 2025ESMC: MLLM-Based Embedding Selection for Explainable Multiple Clustering
  5. 2024Text-Guided Alternative Image Clustering
  6. 2025ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation
  7. 2024Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens
  8. 2026Scalable Semantic Steering of Embedding Projections