# Semantic Diversity Learning for Zero-Shot Multi-label Classification

arXiv: https://arxiv.org/abs/2105.05926 · HTML: https://mlsanity.com/paper/2105.05926/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [Attribute Prototype Network for Zero-Shot Learning](https://mlsanity.com/paper/2008.08290/) (2020)
- [Learning a Deep Embedding Model for Zero-Shot Learning](https://mlsanity.com/paper/1611.05088/) (2016)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Latent Embeddings for Zero-shot Classification](https://mlsanity.com/paper/1603.08895/) (2016)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [TResNet: High Performance GPU-Dedicated Architecture](https://mlsanity.com/paper/2003.13630/) (2020)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Multi-Label Zero-Shot Learning with Structured Knowledge Graphs](https://mlsanity.com/paper/1711.06526/) (2017)
- [Multi-Label Classification with Label Graph Superimposing](https://mlsanity.com/paper/1911.09243/) (2019)
- [Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Concepts](https://mlsanity.com/paper/1803.06049/) (2018)

## Built on by (most cited first, 24 of 24)

- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer](https://mlsanity.com/paper/2207.01887/) (2022)
- [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training](https://mlsanity.com/paper/2312.12828/) (2023)
- [Simple Image-level Classification Improves Open-vocabulary Object Detection](https://mlsanity.com/paper/2312.10439/) (2023)
- [BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classification](https://mlsanity.com/paper/2203.01937/) (2022)
- [CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation](https://mlsanity.com/paper/2406.14830/) (2024)
- [GBE-MLZSL: A Group Bi-Enhancement Framework for Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2309.00923/) (2023)
- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [Semantic Representation and Dependency Learning for Multi-Label Image Recognition](https://mlsanity.com/paper/2204.03795/) (2022)
- [Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Features](https://mlsanity.com/paper/2208.09562/) (2022)
- [Recover and Match: Open-Vocabulary Multi-Label Recognition through Knowledge-Constrained Optimal Transport](https://mlsanity.com/paper/2503.15337/) (2025)
- [Instance-level Few-shot Learning with Class Hierarchy Mining](https://mlsanity.com/paper/2304.07459/) (2023)
- [Text as Image: Learning Transferable Adapter for Multi-Label Classification](https://mlsanity.com/paper/2312.04160/) (2023)
- [Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2408.12253/) (2024)
- [Learning by Asking Questions for Knowledge-based Novel Object Recognition](https://mlsanity.com/paper/2210.05879/) (2022)
- [Query-Based Knowledge Sharing for Open-Vocabulary Multi-Label Classification](https://mlsanity.com/paper/2401.01181/) (2024)
- [Diverse and Tailored Image Generation for Zero-shot Multi-label Classification](https://mlsanity.com/paper/2404.03144/) (2024)
- [Semantic-guided Representation Learning for Multi-Label Recognition](https://mlsanity.com/paper/2504.03801/) (2025)
- [Category-Adaptive Cross-Modal Semantic Refinement and Transfer for Open-Vocabulary Multi-Label Recognition](https://mlsanity.com/paper/2412.06190/) (2024)
- [Multi-label out-of-distribution detection via evidential learning](https://mlsanity.com/paper/2502.18224/) (2025)
- [DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label Recognition](https://mlsanity.com/paper/2508.05585/) (2025)

## Abstract (arXiv)

Training a neural network model for recognizing multiple labels associated with an image, including identifying unseen labels, is challenging, especially for images that portray numerous semantically diverse labels. As challenging as this task is, it is an essential task to tackle since it represents many real-world cases, such as image retrieval of natural images. We argue that using a single embedding vector to represent an image, as commonly practiced, is not sufficient to rank both relevant seen and unseen labels accurately. This study introduces an end-to-end model training for multi-label zero-shot learning that supports semantic diversity of the images and labels. We propose to use an embedding matrix having principal embedding vectors trained using a tailored loss function. In addition, during training, we suggest up-weighting in the loss function image samples presenting higher semantic diversity to encourage the diversity of the embedding matrix. Extensive experiments show that our proposed method improves the zero-shot model's quality in tag-based image retrieval achieving SoTA results on several common datasets (NUS-Wide, COCO, Open Images).