mlsanityData to 7 Oct 2026

Goal-Oriented Gaze Estimation for Zero-Shot Learning

arXiv 2103.03433 ↗First posted 5 Mar 2021cs.CVcited by 68

Not on the map. Why?

Abstract (arXiv)

Zero-shot learning (ZSL) aims to recognize novel classes by transferring semantic knowledge from seen classes to unseen classes. Since semantic knowledge is built on attributes shared between different classes, which are highly local, strong prior for localization of object attribute is beneficial for visual-semantic embedding. Interestingly, when recognizing unseen images, human would also automatically gaze at regions with certain semantic clue. Therefore, we introduce a novel goal-oriented gaze estimation module (GEM) to improve the discriminative attribute localization based on the class-level attributes for ZSL. We aim to predict the actual human gaze location to get the visual attention regions for recognizing a novel object guided by attribute description. Specifically, the task-dependent attention is learned with the goal-oriented GEM, and the global image features are simultaneously optimized with the regression of local attribute features. Experiments on three ZSL benchmarks, i.e., CUB, SUN and AWA2, show the superiority or competitiveness of our proposed method against the state-of-the-art ZSL methods. The ablation analysis on real gaze data CUB-VWSW also validates the benefits and accuracy of our gaze estimation module. This work implies the promising benefits of collecting human gaze dataset and automatic gaze estimation algorithms on high-level computer vision tasks. The code is available at https://github.com/osierboy/GEM-ZSL.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 41 of its 70 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

41 of 70 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  6. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  7. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  8. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  9. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  10. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  11. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  12. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
29 more
  1. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  2. 2016An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wildcited by 146
  3. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  4. 2017Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answeringcited by 117
  5. 2017Generalized Zero-Shot Learning via Synthesized Examplescited by 114
  6. 2018Multi-modal Cycle-consistent Generalized Zero-Shot Learningcited by 87
  7. 2018Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transitioncited by 78
  8. 2020Domain-aware Visual Bias Eliminating for Generalized Zero-Shot Learningcited by 71
  9. 2017Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networkscited by 66
  10. 2019Episode-based Prototype Generating Network for Zero-Shot Learningcited by 66
  11. 2019Transferable Contrastive Network for Generalized Zero-Shot Learningcited by 64
  12. 2020Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learningcited by 64
  13. 2017Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networkscited by 58
  14. 2019Semantic-Guided Multi-Attention Localization for Zero-Shot Learningcited by 58
  15. 2020Generalized Zero-Shot Learning Via Over-Complete Distributioncited by 56
  16. 2019Mutual Context Network for Jointly Estimating Egocentric Gaze and Actionscited by 54
  17. 2020Invertible Zero-Shot Recognition Flowscited by 48
  18. 2018Transductive Unbiased Embedding for Zero-Shot Learningcited by 44
  19. 2016Shallow and Deep Convolutional Networks for Saliency Predictioncited by 42
  20. 2019Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspectivecited by 42
  21. 2018Preserving Semantic Relations for Zero-Shot Learningcited by 35
  22. 2020Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognitioncited by 34
  23. 2019Locality and compositionality in zero-shot learningcited by 29
  24. 2016Gaze Embeddings for Zero-Shot Image Classificationcited by 25
  25. 2018Discriminative Learning of Latent Features for Zero-Shot Recognitioncited by 24
  26. 2018Generalized Zero-Shot Recognition based on Visually Semantic Embeddingcited by 19
  27. 2019Zero-shot Learning via Simultaneous Generating and Learningcited by 17
  28. 2017Semantically Consistent Regularization for Zero-Shot Recognitioncited by 14
  29. 2020Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 16; 2023: 13; 2024: 14; 2025: 11; 2026: 3.

68 papers.

Sort:

By area:

  1. 2021TransZero: Attribute-guided Transformer for Zero-Shot Learningcited by 53
  2. 2022MSDN: Mutually Semantic Distillation Network for Zero-Shot Learningcited by 48
  3. 2022Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learningcited by 36
  4. 2022DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learningcited by 33
  5. 2021TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learningcited by 28
  6. 2022VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learningcited by 27
  7. 2021Dual Progressive Prototype Network for Generalized Zero-Shot Learningcited by 26
  8. 2023Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learningcited by 24
  9. 2023Zero-Shot Learning by Harnessing Adversarial Samplescited by 24
  10. 2022RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classificationcited by 19
  11. 2022Attribute Prototype Network for Any-Shot Learningcited by 19
  12. 2021GaTector: A Unified Framework for Gaze Object Predictioncited by 18
38 more
  1. 2023DRPT: Disentangled and Recurrent Prompt Tuning for Compositional Zero-Shot Learningcited by 17
  2. 2024Progressive Semantic-Guided Vision Transformer for Zero-Shot Learningcited by 17
  3. 2024Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognitioncited by 17
  4. 2024Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learningcited by 16
  5. 2022Semantic-aligned Fusion Transformer for One-shot Object Detectioncited by 13
  6. 2024Fine-Grained Zero-Shot Learning: Advances, Challenges, and Prospectscited by 12
  7. 2024CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learningcited by 12
  8. 2021Disentangling Semantic-to-visual Confusion for Zero-shot Learningcited by 11
  9. 2023Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worldscited by 11
  10. 2021What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Datacited by 10
  11. 2025Where, What, Why: Towards Explainable Driver Attention Predictioncited by 10
  12. 2023Multi-Semantic Fusion Model for Generalized Zero-Shot Skeleton-Based Action Recognitioncited by 9
  13. 2022Boosting Zero-shot Learning via Contrastive Optimization of Attribute Representationscited by 8
  14. 2022Rebalanced Zero-shot Learningcited by 8
  15. 2021Region Semantically Aligned Network for Zero-Shot Learningcited by 6
  16. 2022Deconstructed Generation-Based Zero-Shot Modelcited by 6
  17. 2025SVIP: Semantically Contextualized Visual Patches for Zero-Shot Learningcited by 6
  18. 2021Implicit and Explicit Attention for Zero-Shot Learningcited by 5
  19. 2021Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Datacited by 5
  20. 2024AlignZeg: Mitigating Objective Misalignment for Zero-shot Semantic Segmentationcited by 5
  21. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  22. 2025GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencodercited by 5
  23. 2021Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classificationcited by 4
  24. 2022Zero-Shot Classification by Logical Reasoning on Natural Language Explanationscited by 4
  25. 2024Data Distribution Distilled Generative Model for Generalized Zero-Shot Recognitioncited by 4
  26. 2025Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learningcited by 4
  27. 2023An Integral Projection-based Semantic Autoencoder for Zero-Shot Learningcited by 3
  28. 2023HOPE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Expertscited by 3
  29. 2024Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classificationcited by 3
  30. 2024PSVMA+: Exploring Multi-granularity Semantic-visual Adaption for Generalized Zero-shot Learningcited by 3
  31. 2025Distributed Zero-Shot Learning for Visual Recognitioncited by 3
  32. 2023Zero-Knowledge Zero-Shot Learning for Novel Visual Category Discoverycited by 2
  33. 2025MADS: Multi-Attribute Document Supervision for Zero-Shot Image Classificationcited by 2
  34. 2022Learning Compositional Representations for Effective Low-Shot Generalizationcited by 1
  35. 2022Where and What: Driver Attention-based Object Detectioncited by 1
  36. 2022Learning Prototype via Placeholder for Zero-shot Recognitioncited by 1
  37. 2023GazeCLIP: Enhancing Gaze Estimation Through Text-Guided Multimodal Learningcited by 1
  38. 2024`Eyes of a Hawk and Ears of a Fox': Part Prototype Network for Generalized Zero-Shot Learningcited by 1

50 most cited of 68 All 68 (JSON)