mlsanityData to 14 Oct 2026

Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning

arXiv 2404.07713 ↗First posted 11 Apr 2024cs.CV cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Zero-shot learning (ZSL) recognizes the unseen classes by conducting visual-semantic interactions to transfer semantic knowledge from seen classes to unseen ones, supported by semantic information (e.g., attributes). However, existing ZSL methods simply extract visual features using a pre-trained network backbone (i.e., CNN or ViT), which fail to learn matched visual-semantic correspondences for representing semantic-related visual features as lacking of the guidance of semantic information, resulting in undesirable visual-semantic interactions. To tackle this issue, we propose a progressive semantic-guided vision transformer for zero-shot learning (dubbed ZSLViT). ZSLViT mainly considers two properties in the whole network: i) discover the semantic-related visual representations explicitly, and ii) discard the semantic-unrelated visual information. Specifically, we first introduce semantic-embedded token learning to improve the visual-semantic correspondences via semantic enhancement and discover the semantic-related visual tokens explicitly with semantic-guided token attention. Then, we fuse low semantic-visual correspondence visual tokens to discard the semantic-unrelated visual information for visual enhancement. These two operations are integrated into various encoders to progressively learn semantic-related visual representations for accurate visual-semantic interactions in ZSL. The extensive experiments show that our ZSLViT achieves significant performance gains on three popular benchmark datasets, i.e., CUB, SUN, and AWA2. Codes are available at: https://github.com/shiming-chen/ZSLViT .

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

44 of 58 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  12. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
32 more
  1. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  2. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 255
  3. 2019Local Relation Networks for Image Recognitioncited by 210
  4. 2015Label-Embedding for Image Classificationcited by 165
  5. 2019f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learningcited by 164
  6. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
  7. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalizationcited by 101
  8. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  9. 2021Contrastive Embedding for Generalized Zero-Shot Learningcited by 94
  10. 2021Goal-Oriented Gaze Estimation for Zero-Shot Learningcited by 68
  11. 2021FREE: Feature Refinement for Generalized Zero-Shot Learningcited by 67
  12. 2020Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learningcited by 64
  13. 2019Semantic-Guided Multi-Attention Localization for Zero-Shot Learningcited by 58
  14. 2021HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learningcited by 57
  15. 2021TransZero: Attribute-guided Transformer for Zero-Shot Learningcited by 54
  16. 2022I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classificationcited by 54
  17. 2023Improving Zero-Shot Generalization for CLIP with Synthesized Promptscited by 51
  18. 2022MSDN: Mutually Semantic Distillation Network for Zero-Shot Learningcited by 48
  19. 2019Transductive Zero-Shot Learning with Visual Structure Constraintcited by 36
  20. 2022DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learningcited by 33
  21. 2021TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learningcited by 28
  22. 2021Discriminative Region-based Multi-Label Zero-Shot Learningcited by 27
  23. 2022I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classificationcited by 27
  24. 2023Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learningcited by 24
  25. 2023Evolving Semantic Prototype Improves Generative Zero-Shot Learningcited by 20
  26. 2021Generative Multi-Label Zero-Shot Learningcited by 19
  27. 2023Image-free Classifier Injection for Zero-Shot Classificationcited by 15
  28. 2021Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learningcited by 12
  29. 2021Semantic Feature Extraction for Generalized Zero-shot Learningcited by 7
  30. 2023EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learningcited by 6
  31. 2021Implicit and Explicit Attention for Zero-Shot Learningcited by 5
  32. 2023Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignmentcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 3.

17 papers.

Sort:

By area:

  1. 2024ZeroDiff: Solidified Visual-Semantic Correlation in Zero-Shot Learningcited by 10
  2. 2025SVIP: Semantically Contextualized Visual Patches for Zero-Shot Learningcited by 6
  3. 2025Interpretable Zero-Shot Learning with Locally-Aligned Vision-Language Modelcited by 6
  4. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  5. 2025GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencodercited by 5
  6. 2025Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learningcited by 4
  7. 2024Visual-Semantic Graph Matching Net for Zero-Shot Learningcited by 3
  8. 2025MADS: Multi-Attribute Document Supervision for Zero-Shot Image Classificationcited by 2
  9. 2025Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidancecited by 1
  10. 2025Interpretable Zero-shot Learning with Infinite Class Conceptscited by 1
  11. 2025Semantic-Inductive Attribute Selection for Zero-Shot Learningcited by 0
  12. 2025Multi-Granularity Mutual Refinement Network for Zero-Shot Learningcited by 0
5 more
  1. 2025Fine-Grained Zero-Shot Learning with Attribute-Centric Representationscited by 0
  2. 2025Adversarial Robustness in Zero-Shot Learning:An Empirical Study on Class and Concept-Level Vulnerabilitiescited by 0
  3. 2026ZeroDiff++: Substantial Unseen Visual-semantic Correlation in Zero-shot Learningcited by 0
  4. 2026Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learningcited by 0
  5. 2026Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cuescited by 0