# Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning arXiv: https://arxiv.org/abs/2408.12253 ยท HTML: https://mlsanity.com/paper/2408.12253/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 46 of its 77 references that are in the map. - First posted on arXiv 22 Aug 2024. ## Builds on (most cited first, 30 of 46) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014) - [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019) - [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018) - [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017) - [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016) - [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015) - [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013) - [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019) - [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017) - [Multi-Label Zero-Shot Learning with Structured Knowledge Graphs](https://mlsanity.com/paper/1711.06526/) (2017) - [Large-scale Multi-label Learning with Missing Labels](https://mlsanity.com/paper/1307.5101/) (2013) - [Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition](https://mlsanity.com/paper/2007.01755/) (2020) - [Deep Convolutional Ranking for Multilabel Image Annotation](https://mlsanity.com/paper/1312.4894/) (2013) - [DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning](https://mlsanity.com/paper/2403.08506/) (2024) - [A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion](https://mlsanity.com/paper/2004.14795/) (2020) - [SRCD: Semantic Reasoning with Compound Domains for Single-Domain Generalized Object Detection](https://mlsanity.com/paper/2307.01750/) (2023) - [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021) - [Semantic Diversity Learning for Zero-Shot Multi-label Classification](https://mlsanity.com/paper/2105.05926/) (2021) - [Personalized Federated Learning with Contextualized Generalization](https://mlsanity.com/paper/2106.13044/) (2021) - [CaDM: Codec-aware Diffusion Modeling for Neural-enhanced Video Streaming](https://mlsanity.com/paper/2211.08428/) (2022) - [Multi-Cue Zero-Shot Learning with Strong Supervision](https://mlsanity.com/paper/1603.08754/) (2016) - [Fast Zero-Shot Image Tagging](https://mlsanity.com/paper/1605.09759/) (2016) - [MlTr: Multi-label Classification with Transformer](https://mlsanity.com/paper/2106.06195/) (2021) - [Towards Unbiased Multi-label Zero-Shot Learning with Pyramid and Semantic Attention](https://mlsanity.com/paper/2203.03483/) (2022) - [Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matching](https://mlsanity.com/paper/2407.05005/) (2024) - [AMS-SFE: Towards an Alignment of Manifold Structures via Semantic Feature Expansion for Zero-shot Learning](https://mlsanity.com/paper/1904.06254/) (2019) ## Built on by (most cited first, 3 of 3) - [Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2411.11288/) (2024) - [Dual-View Alignment Learning with Hierarchical-Prompt for Class-Imbalance Multi-Label Classification](https://mlsanity.com/paper/2509.17747/) (2025) - [FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data](https://mlsanity.com/paper/2502.07456/) (2025) ## Abstract (arXiv) This paper investigates a challenging problem of zero-shot learning in the multi-label scenario (MLZSL), wherein the model is trained to recognize multiple unseen classes within a sample (e.g., an image) based on seen classes and auxiliary knowledge, e.g., semantic information. Existing methods usually resort to analyzing the relationship of various seen classes residing in a sample from the dimension of spatial or semantic characteristics and transferring the learned model to unseen ones. However, they neglect the integrity of local and global features. Although the use of the attention structure will accurately locate local features, especially objects, it will significantly lose its integrity, and the relationship between classes will also be affected. Rough processing of global features will also directly affect comprehensiveness. This neglect will make the model lose its grasp of the main components of the image. Relying only on the local existence of seen classes during the inference stage introduces unavoidable bias. In this paper, we propose a novel and comprehensive visual-semantic framework for MLZSL, dubbed Epsilon, to fully make use of such properties and enable a more accurate and robust visual-semantic projection. In terms of spatial information, we achieve effective refinement by group aggregating image features into several semantic prompts. It can aggregate semantic information rather than class information, preserving the correlation between semantics. In terms of global semantics, we use global forward propagation to collect as much information as possible to ensure that semantics are not omitted. Experiments on large-scale MLZSL benchmark datasets NUS-Wide and Open-Images-v4 demonstrate that the proposed Epsilon outperforms other state-of-the-art methods with large margins.