mlsanityData to 7 Oct 2026

Large Language Models are Good Prompt Learners for Low-Shot Image Classification

arXiv 2312.04076 ↗First posted 7 Dec 2023cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Low-shot image classification, where training images are limited or inaccessible, has benefited from recent progress on pre-trained vision-language (VL) models with strong generalizability, e.g. CLIP. Prompt learning methods built with VL models generate text features from the class names that only have confined class-specific information. Large Language Models (LLMs), with their vast encyclopedic knowledge, emerge as the complement. Thus, in this paper, we discuss the integration of LLMs to enhance pre-trained VL models, specifically on low-shot classification. However, the domain gap between language and vision blocks the direct application of LLMs. Thus, we propose LLaMP, Large Language Models as Prompt learners, that produces adaptive prompts for the CLIP text encoder, establishing it as the connecting bridge. Experiments show that, compared with other state-of-the-art prompt learning methods, LLaMP yields better performance on both zero-shot generalization and few-shot image classification, over a spectrum of 11 datasets. Code will be made available at: https://github.com/zhaohengz/LLaMP.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

35 of 50 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
23 more
  1. 2013Describing Textures in the Wildcited by 2,110
  2. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  3. 2022Visual Prompt Tuningcited by 1,674
  4. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  5. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  6. 2023PaLM 2 Technical Reportcited by 1,154
  7. 2021Learning to Prompt for Continual Learningcited by 938
  8. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  9. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  10. 2022Efficiently Scaling Transformer Inferencecited by 529
  11. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  12. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  13. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  14. 2022Prompt Distribution Learningcited by 235
  15. 2016Learning Deep Representations of Fine-grained Visual Descriptionscited by 182
  16. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  17. 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
  18. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  19. 2017A Generative Adversarial Approach for Zero-Shot Learning from Noisy Textscited by 96
  20. 2018Rethinking Knowledge Graph Propagation for Zero-Shot Learningcited by 87
  21. 2021FREE: Feature Refinement for Generalized Zero-Shot Learningcited by 67
  22. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
  23. 2023Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Modelscited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 12; 2026: 5.

21 papers.

Sort:

By area:

  1. 2024Tree of Attributes Prompt Learning for Vision-Language Modelscited by 7
  2. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  3. 2024TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaborationcited by 5
  4. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classificationcited by 5
  5. 2025Provably Improving Generalization of Few-Shot Models with Synthetic Datacited by 3
  6. 2025Free-Grained Hierarchical Visual Recognitioncited by 3
  7. 2025Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Modelscited by 2
  8. 2025Decoupling Continual Semantic Segmentationcited by 2
  9. 2025Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconographycited by 2
  10. 2025Anatomy-Aware Text-Visual Fusion with Dual-Perspective Prompts for Fine-Grained Lumbar Spine Segmentationcited by 1
  11. 2025LLM-enhanced Action-aware Multi-modal Prompt Tuning for Image-Text Matchingcited by 1
  12. 2025UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrievalcited by 1
9 more
  1. 2025DiVE-k: Differential Visual Reasoning for Fine-grained Image Recognitioncited by 1
  2. 2025AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learningcited by 1
  3. 2026Prompt Tuning for CLIP on the Pretrained Manifoldcited by 1
  4. 2026DeAR: Fine-Grained VLM Adaptation by Decomposing Attention Head Rolescited by 1
  5. 2024If you can describe it, they can see it: Cross-Modal Learning of Visual Concepts from Textual Descriptionscited by 0
  6. 2025Decoding Neighborhood Environments with Large Language Modelscited by 0
  7. 2026MultiModal Fine-tuning with Synthetic Captionscited by 0
  8. 2026SpecPL: Disentangling Spectral Granularity for Prompt Learningcited by 0
  9. 2026Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AIcited by 0