mlsanityData to 7 Oct 2026

Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions

arXiv 2207.03133 ↗First posted 7 Jul 2022cs.CL cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Humans can obtain the knowledge of novel visual concepts from language descriptions, and we thus use the few-shot image classification task to investigate whether a machine learning model can have this capability. Our proposed model, LIDE (Learning from Image and DEscription), has a text decoder to generate the descriptions and a text encoder to obtain the text representations of machine- or user-generated descriptions. We confirmed that LIDE with machine-generated descriptions outperformed baseline models. Moreover, the performance was improved further with high-quality user-generated descriptions. The generated descriptions can be viewed as the explanations of the model's predictions, and we observed that such explanations were consistent with prediction results. We also investigated why the language description improved the few-shot image classification performance by comparing the image representations and the text representations in the feature spaces.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 7 Jul 2022.

Builds on

24 of 51 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2016Layer Normalizationcited by 5,579
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  11. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  12. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
12 more
  1. 2016Learning Deep Representations of Fine-grained Visual Descriptionscited by 182
  2. 2016Generating Visual Explanationscited by 156
  3. 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
  4. 2018Grounding Visual Explanationscited by 92
  5. 2017Learning with Latent Languagecited by 75
  6. 2017Fine-graind Image Classification via Combining Vision and Languagecited by 51
  7. 2020ALICE: Active Learning with Contrastive Natural Language Explanationscited by 36
  8. 2019Shaping Visual Representations with Language for Few-shot Classificationcited by 27
  9. 2018Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captionscited by 22
  10. 2017Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captionercited by 21
  11. 2021Rich Semantics Improve Few-shot Learningcited by 15
  12. 2021Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classificationcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 4; 2025: 1.

6 papers.

  1. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  2. 2024GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpathscited by 15
  3. 2024Explanation Bottleneck Modelscited by 4
  4. 2025Coreset Selection via LLM-based Concept Bottleneckscited by 2
  5. 2024LLM-based Hierarchical Concept Decomposition for Interpretable Fine-Grained Image Classificationcited by 1
  6. 2024Reading Is Believing: Revisiting Language Bottleneck Models for Image Classificationcited by 0