# Large-Scale Zero-Shot Image Classification from Rich and Diverse Textual Descriptions

arXiv: https://arxiv.org/abs/2103.09669 · HTML: https://mlsanity.com/paper/2103.09669/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [VirTex: Learning Visual Representations from Textual Annotations](https://mlsanity.com/paper/2006.06666/) (2020)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning](https://mlsanity.com/paper/1903.10132/) (2019)
- [Yin and Yang: Balancing and Answering Binary Visual Questions](https://mlsanity.com/paper/1511.05099/) (2015)
- [A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts](https://mlsanity.com/paper/1712.01381/) (2017)
- [Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precision](https://mlsanity.com/paper/1709.01148/) (2017)
- [CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language](https://mlsanity.com/paper/1909.09822/) (2019)
- [Write a Classifier: Predicting Visual Classifiers from Unstructured Text](https://mlsanity.com/paper/1601.00025/) (2015)

## Built on by (most cited first, 21 of 21)

- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [What does a platypus look like? Generating customized prompts for zero-shot image classification](https://mlsanity.com/paper/2209.03320/) (2022)
- [Why are Visually-Grounded Language Models Bad at Image Classification?](https://mlsanity.com/paper/2405.18415/) (2024)
- [Waffling around for Performance: Visual Classification with Random Words and Broad Concepts](https://mlsanity.com/paper/2306.07282/) (2023)
- [CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets](https://mlsanity.com/paper/2302.02551/) (2023)
- [Open-Set Image Tagging with Multi-Grained Text Supervision](https://mlsanity.com/paper/2310.15200/) (2023)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)
- [I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification](https://mlsanity.com/paper/2212.02291/) (2022)
- [I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification](https://mlsanity.com/paper/2209.10304/) (2022)
- [Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images](https://mlsanity.com/paper/2311.01064/) (2023)
- [Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings](https://mlsanity.com/paper/2202.02002/) (2022)
- [Visual-Semantic Decomposition and Partial Alignment for Document-based Zero-Shot Learning](https://mlsanity.com/paper/2407.15613/) (2024)
- [Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP](https://mlsanity.com/paper/2410.23698/) (2024)
- [SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalization](https://mlsanity.com/paper/2202.13100/) (2022)
- [M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios](https://mlsanity.com/paper/2303.05122/) (2023)
- [Coreset Selection via LLM-based Concept Bottlenecks](https://mlsanity.com/paper/2502.16733/) (2025)
- [MADS: Multi-Attribute Document Supervision for Zero-Shot Image Classification](https://mlsanity.com/paper/2503.06847/) (2025)
- [Text2Model: Text-based Model Induction for Zero-shot Image Classification](https://mlsanity.com/paper/2210.15182/) (2022)
- [LLM-based Hierarchical Concept Decomposition for Interpretable Fine-Grained Image Classification](https://mlsanity.com/paper/2405.18672/) (2024)
- [Towards Efficient and General-Purpose Few-Shot Misclassification Detection for Vision-Language Models](https://mlsanity.com/paper/2503.20492/) (2025)
- [Describe me an Aucklet: Generating Grounded Perceptual Category Descriptions](https://mlsanity.com/paper/2303.04053/) (2023)

## Abstract (arXiv)

We study the impact of using rich and diverse textual descriptions of classes for zero-shot learning (ZSL) on ImageNet. We create a new dataset ImageNet-Wiki that matches each ImageNet class to its corresponding Wikipedia article. We show that merely employing these Wikipedia articles as class descriptions yields much higher ZSL performance than prior works. Even a simple model using this type of auxiliary data outperforms state-of-the-art models that rely on standard features of word embedding encodings of class names. These results highlight the usefulness and importance of textual descriptions for ZSL, as well as the relative importance of auxiliary data type compared to algorithmic progress. Our experimental results also show that standard zero-shot learning approaches generalize poorly across categories of classes.