mlsanityData to 7 Oct 2026

Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly

arXiv 1707.00600 ↗First posted 3 Jul 2017cs.CVcited by 536

Not on the map. Why?

Cited by 20 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Due to the importance of zero-shot learning, i.e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily. We argue that it is time to take a step back and to analyze the status quo of the area. The purpose of this paper is three-fold. First, given the fact that there is no agreed upon zero-shot learning benchmark, we first define a new benchmark by unifying both the evaluation protocols and data splits of publicly available datasets used for this task. This is an important contribution as published results are often not comparable and sometimes even flawed due to, e.g. pre-training on zero-shot test classes. Moreover, we propose a new zero-shot learning dataset, the Animals with Attributes 2 (AWA2) dataset which we make publicly available both in terms of image features and the images themselves. Second, we compare and analyze a significant number of the state-of-the-art methods in depth, both in the classic zero-shot setting but also in the more realistic generalized zero-shot setting. Finally, we discuss in detail the limitations of the current status of the area which can be taken as a basis for advancing it.

Key facts. Cited by 536 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 126; 2022: 117; 2023: 94; 2024: 103; 2025: 69; 2026: 26.

536 papers.

Sort:

By area:

  1. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  2. 2021Domain Generalization: A Surveycited by 760
  3. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  4. 2022Multimodal Learning with Transformers: A Surveycited by 399
  5. 2022Visual Classification via Description from Large Language Modelscited by 331
  6. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  7. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  8. 2021Decoupling Zero-Shot Semantic Segmentationcited by 198
  9. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  10. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  11. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  12. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
38 more
  1. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 140
  2. 2021Graph Learning: A Surveycited by 140
  3. 2022Linearly Mapping from Image to Text Spacecited by 139
  4. 2021AP-10K: A Benchmark for Animal Pose Estimation in the Wildcited by 123
  5. 2021AutoNovel: Automatically Discovering and Learning Novel Visual Categoriescited by 116
  6. 2021Learning Graph Embeddings for Compositional Zero-shot Learningcited by 112
  7. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  8. 2022Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understandingcited by 102
  9. 2023Probabilistic Concept Bottleneck Modelscited by 101
  10. 2021Open World Compositional Zero-Shot Learningcited by 100
  11. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  12. 2021Learning to Predict Visual Attributes in the Wildcited by 95
  13. 2021Contrastive Embedding for Generalized Zero-Shot Learningcited by 94
  14. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  15. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  16. 2024Robust agents learn causal world modelscited by 81
  17. 2021Detecting Human-Object Interaction via Fabricated Compositional Learningcited by 79
  18. 2022K-LITE: Learning Transferable Visual Models with External Knowledgecited by 79
  19. 2021Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformerscited by 76
  20. 2021Elaborative Rehearsal for Zero-shot Action Recognitioncited by 76
  21. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  22. 2021Goal-Oriented Gaze Estimation for Zero-Shot Learningcited by 68
  23. 2022Learning to Compose Soft Prompts for Compositional Zero-Shot Learningcited by 66
  24. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  25. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  26. 2022Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Featurescited by 63
  27. 2023Frozen Language Model Helps ECG Zero-Shot Learningcited by 62
  28. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  29. 2024Learning the 3D Fauna of the Webcited by 58
  30. 2021A Closer Look at Self-training for Zero-Label Semantic Segmentationcited by 57
  31. 2023Zero-shot Object Countingcited by 57
  32. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  33. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  34. 2022Siamese Contrastive Embedding Network for Compositional Zero-Shot Learningcited by 55
  35. 2022I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classificationcited by 54
  36. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  37. 2023COLA: A Benchmark for Compositional Text-to-image Retrievalcited by 53
  38. 2023Improving Zero-Shot Generalization for CLIP with Synthesized Promptscited by 51

50 most cited of 536 All 536 (JSON)