mlsanityData to 14 Oct 2026

I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification

arXiv 2212.02291 ↗First posted 5 Dec 2022cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent works have shown that unstructured text (documents) from online sources can serve as useful auxiliary information for zero-shot image classification. However, these methods require access to a high-quality source like Wikipedia and are limited to a single source of information. Large Language Models (LLM) trained on web-scale text show impressive abilities to repurpose their learned knowledge for a multitude of tasks. In this work, we provide a novel perspective on using an LLM to provide text supervision for a zero-shot image classification model. The LLM is provided with a few text descriptions from different annotators as examples. The LLM is conditioned on these examples to generate multiple text descriptions for each class(referred to as views). Our proposed model, I2MVFormer, learns multi-view semantic embeddings for zero-shot image classification with these class views. We show that each text view of a class provides complementary information allowing a model to learn a highly discriminative class embedding. Moreover, we show that I2MVFormer is better at consuming the multi-view text supervision from LLM compared to baseline models. I2MVFormer establishes a new state-of-the-art on three public benchmark datasets for zero-shot image classification with unsupervised semantic embeddings.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 51 of its 66 references that are in the map. First posted on arXiv 5 Dec 2022.

Builds on

51 of 66 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2020Longformer: The Long-Document Transformercited by 3,413
  10. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  11. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  12. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
38 more
  1. 2021Learning to Prompt for Continual Learningcited by 944
  2. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  3. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  4. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  5. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
  6. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  7. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  8. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  9. 2022Prompt Distribution Learningcited by 235
  10. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  11. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  12. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
  13. 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
  14. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  15. 2016An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wildcited by 146
  16. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  17. 2017Generalized Zero-Shot Learning via Synthesized Examplescited by 114
  18. 2016Synthesized Classifiers for Zero-Shot Learningcited by 113
  19. 2021Learning Graph Embeddings for Compositional Zero-shot Learningcited by 112
  20. 2021Open World Compositional Zero-Shot Learningcited by 101
  21. 2017A Generative Adversarial Approach for Zero-Shot Learning from Noisy Textscited by 96
  22. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  23. 2018Rethinking Knowledge Graph Propagation for Zero-Shot Learningcited by 87
  24. 2019Transferable Contrastive Network for Generalized Zero-Shot Learningcited by 64
  25. 2019Semantic-Guided Multi-Attention Localization for Zero-Shot Learningcited by 58
  26. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  27. 2021Learning Graph Embeddings for Open World Compositional Zero-Shot Learningcited by 49
  28. 2017Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precisioncited by 46
  29. 2016Less is more: zero-shot learning from online textual documents with noise suppressioncited by 40
  30. 2019Learning Feature-to-Feature Translator by Alternating Back-Propagation for Generative Zero-Shot Learningcited by 38
  31. 2017Generating Visual Representations for Zero-Shot Classificationcited by 37
  32. 2022VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learningcited by 27
  33. 2022I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classificationcited by 27
  34. 2021Large-Scale Zero-Shot Image Classification from Rich and Diverse Textual Descriptionscited by 21
  35. 2022Learning Attention Propagation for Compositional Zero-Shot Learningcited by 20
  36. 20213D Compositional Zero-shot Learning with DeCompositional Consensuscited by 16
  37. 2017Automatic Discovery, Association Estimation and Learning of Semantic Attributes for a Thousand Categoriescited by 10
  38. 2018Selective Zero-Shot Classification with Augmented Attributescited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 20; 2025: 18; 2026: 5.

54 papers.

Sort:

By area:

  1. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 101
  2. 2023SILC: Improving Vision Language Pretraining with Self-Distillationcited by 66
  3. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptionscited by 54
  4. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Promptscited by 50
  5. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  6. 2024UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them Allcited by 49
  7. 2023SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidancecited by 41
  8. 2024Learning to Prompt with Text Only Supervision for Vision-Language Modelscited by 38
  9. 2024Agent3D-Zero: An Agent for Zero-shot 3D Understandingcited by 29
  10. 2024Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detectioncited by 25
  11. 2025Seek-CAD: A Self-refined Generative Modeling for 3D Parametric CAD Using Local Inference via DeepSeekcited by 24
  12. 2024Progressive Semantic-Guided Vision Transformer for Zero-Shot Learningcited by 17
38 more
  1. 2024Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learningcited by 16
  2. 2023Image-free Classifier Injection for Zero-Shot Classificationcited by 15
  3. 2024Open-Vocabulary Calibration for Fine-tuned CLIPcited by 15
  4. 2024ExpLLM: Towards Chain of Thought for Facial Expression Recognitioncited by 15
  5. 2025ProAPO: Progressively Automatic Prompt Optimization for Visual Classificationcited by 13
  6. 2023Integrating Language-Derived Appearance Elements with Visual Cues in Pedestrian Detectioncited by 10
  7. 2024Context-Enhanced Video Moment Retrieval with Large Language Modelscited by 9
  8. 2024Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Modelscited by 9
  9. 2023Localizing Active Objects from Egocentric Vision with Symbolic World Knowledgecited by 8
  10. 2023A Survey on Open-Set Image Recognitioncited by 7
  11. 2024LLMBind: A Unified Modality-Task Integration Frameworkcited by 7
  12. 2025AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysiscited by 7
  13. 2025SVIP: Semantically Contextualized Visual Patches for Zero-Shot Learningcited by 6
  14. 2025GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencodercited by 5
  15. 2025SpatialMosaic: A Multiview VLM Dataset for Partial Visibilitycited by 5
  16. 2023Open-Pose 3D Zero-Shot Learning: Benchmark and Challengescited by 4
  17. 2024ECOR: Explainable CLIP for Object Recognitioncited by 4
  18. 2024Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Modelcited by 4
  19. 2023Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignmentcited by 3
  20. 2024Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Taskscited by 3
  21. 2024MSCoTDet: Language-driven Multi-modal Fusion for Improved Multispectral Pedestrian Detectioncited by 3
  22. 2025SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detectioncited by 3
  23. 2024Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrievalcited by 2
  24. 2025Generate, Transduct, Adapt: Iterative Transduction with VLMscited by 2
  25. 2025MADS: Multi-Attribute Document Supervision for Zero-Shot Image Classificationcited by 2
  26. 2026Dual-Modality Anchor-Guided Filtering for Test-time Prompt Tuningcited by 2
  27. 2024`Eyes of a Hawk and Ears of a Fox': Part Prototype Network for Generalized Zero-Shot Learningcited by 1
  28. 2024CICA: Content-Injected Contrastive Alignment for Zero-Shot Document Image Classificationcited by 1
  29. 2025Interpretable Zero-shot Learning with Infinite Class Conceptscited by 1
  30. 2026NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inferencecited by 1
  31. 2026AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMscited by 1
  32. 2024Enabling Generalized Zero-shot Learning Towards Unseen Domains by Intrinsic Learning from Redundant LLM Semanticscited by 0
  33. 2025Memory-Modular Classification: Learning to Generalize with Memory Replacementcited by 0
  34. 2025Neural Catalog: Scaling Species Recognition with Catalog of Life-Augmented Generationcited by 0
  35. 2025Set Pivot Learning: Redefining Generalized Segmentation with Vision Foundation Modelscited by 0
  36. 2025No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learningcited by 0
  37. 2025Continual Learning for Image Captioning through Improved Image-Text Alignmentcited by 0
  38. 2025Dynamic Multi-level Weighted Alignment Network for Zero-shot Sketch-based Image Retrievalcited by 0

50 most cited of 54 All 54 (JSON)