mlsanityData to 7 Oct 2026

ChatGPT-Powered Hierarchical Comparisons for Image Classification

arXiv 2311.00206 ↗First posted 1 Nov 2023cs.CV cs.AIcited by 22

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

The zero-shot open-vocabulary challenge in image classification is tackled by pretrained vision-language models like CLIP, which benefit from incorporating class-specific knowledge from large language models (LLMs) like ChatGPT. However, biases in CLIP lead to similar descriptions for distinct but related classes, prompting our novel image classification framework via hierarchical comparisons: using LLMs to recursively group classes into hierarchies and classifying images by comparing image-text embeddings at each hierarchy level, resulting in an intuitive, effective, and explainable approach.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 52 references that are in the map. First posted on arXiv 1 Nov 2023.

Builds on

38 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Visual Classification via Description from Large Language Models13 Oct
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  3. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  4. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct

Other references · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
22 more
  1. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  2. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  3. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  4. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  5. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  6. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  7. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  8. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  9. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  10. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  11. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  12. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  13. 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
  14. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  15. 2023CHiLS: Zero-Shot Image Classification with Hierarchical Label Setscited by 79
  16. 2019Deep Tree Learning for Zero-shot Face Anti-Spoofingcited by 73
  17. 2019Semantic-Aware Scene Recognitioncited by 43
  18. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Modelcited by 41
  19. 2021Unified Detection of Digital and Physical Face Attackscited by 31
  20. 2022Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classificationcited by 21
  21. 2023TransHP: Image Classification with Hierarchical Promptingcited by 16
  22. 20220/1 Deep Neural Networks via Block Coordinate Descentcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 9; 2026: 6.

22 papers.

Sort:

By area:

  1. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 51
  2. 2023GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition?cited by 33
  3. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Modelscited by 20
  4. 2025BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Modelscited by 18
  5. 2024Open-Set Biometrics: Beyond Good Closed-Set Modelscited by 12
  6. 2025ProAPO: Progressively Automatic Prompt Optimization for Visual Classificationcited by 12
  7. 2024GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognitioncited by 8
  8. 2025Enhancing Concept Localization in CLIP-based Concept Bottleneck Modelscited by 6
  9. 2025BatStyler: Advancing Multi-category Style Generation for Source-free Domain Generalizationcited by 4
  10. 2025AgriRegion: Region-Aware Retrieval for High-Fidelity Agricultural Advicecited by 4
  11. 2026LocalScore: Local Density-Aware Similarity Scoring for Biometricscited by 3
  12. 2025Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Promptscited by 2
10 more
  1. 2024Grounding Descriptions in Images informs Zero-Shot Visual Recognitioncited by 1
  2. 2025Edge Attention Module for Object Classificationcited by 1
  3. 2024Analysis of Unstructured High-Density Crowded Scenes for Crowd Monitoringcited by 0
  4. 2025Decoding Neighborhood Environments with Large Language Modelscited by 0
  5. 2025Beyond Class Tokens: LLM-guided Dominant Property Mining for Few-shot Classificationcited by 0
  6. 2026RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMscited by 0
  7. 2026Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomycited by 0
  8. 2026AdaBoosting Text Prompts for Vision-Language Modelscited by 0
  9. 2026Subtoken Vision Transformer for Fine-grained Recognitioncited by 0
  10. 2026Position Aware Layer Queries for Test Time Training in Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Visual Classification using Comparative Descriptors
  2. 2023Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification
  3. 2023HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding
  4. 2026Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings
  5. 2023LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptions
  6. 2023Vocabulary-free Image Classification
  7. 2024Revisiting MLLMs: An In-Depth Analysis of Image Classification Abilities
  8. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Concepts