# Vocabulary-informed Zero-shot and Open-set Learning

arXiv: https://arxiv.org/abs/2301.00998 · HTML: https://mlsanity.com/paper/2301.00998/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 100 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 30 of 32)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [Evaluation of Output Embeddings for Fine-Grained Image Classification](https://mlsanity.com/paper/1409.8403/) (2014)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [Learning a Deep Embedding Model for Zero-Shot Learning](https://mlsanity.com/paper/1611.05088/) (2016)
- [An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild](https://mlsanity.com/paper/1605.04253/) (2016)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Latent Embeddings for Zero-shot Classification](https://mlsanity.com/paper/1603.08895/) (2016)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [Generalized Zero-Shot Learning via Synthesized Examples](https://mlsanity.com/paper/1712.03878/) (2017)
- [Synthesized Classifiers for Zero-Shot Learning](https://mlsanity.com/paper/1603.00550/) (2016)
- [Multi-modal Cycle-consistent Generalized Zero-Shot Learning](https://mlsanity.com/paper/1808.00136/) (2018)
- [The Extreme Value Machine](https://mlsanity.com/paper/1506.06112/) (2015)
- [Zero-Shot Learning via Semantic Similarity Embedding](https://mlsanity.com/paper/1509.04767/) (2015)
- [Improving zero-shot learning by mitigating the hubness problem](https://mlsanity.com/paper/1412.6568/) (2014)
- [Transductive Multi-view Zero-Shot Learning](https://mlsanity.com/paper/1501.04560/) (2015)
- [Ridge Regression, Hubness, and Zero-Shot Learning](https://mlsanity.com/paper/1507.00825/) (2015)
- [Zero Shot Recognition with Unreliable Attributes](https://mlsanity.com/paper/1409.4327/) (2014)
- [A Simple Exponential Family Framework for Zero-Shot Learning](https://mlsanity.com/paper/1707.08040/) (2017)
- [Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning](https://mlsanity.com/paper/1605.08151/) (2016)
- [Recent Advances in Zero-shot Recognition](https://mlsanity.com/paper/1710.04837/) (2017)
- [From Zero-shot Learning to Conventional Supervised Classification: Unseen Visual Data Synthesis](https://mlsanity.com/paper/1705.01782/) (2017)
- [Semi-supervised Vocabulary-informed Learning](https://mlsanity.com/paper/1604.07093/) (2016)
- [A Unified Semantic Embedding: Relating Taxonomies and Attributes](https://mlsanity.com/paper/1411.5879/) (2014)

## Built on by (most cited first, 4 of 4)

- [Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worlds](https://mlsanity.com/paper/2307.03416/) (2023)
- [Hybrid Open-set Segmentation with Synthetic Negative Data](https://mlsanity.com/paper/2301.08555/) (2023)
- [Benchmarking Object Detectors under Real-World Distribution Shifts in Satellite Imagery](https://mlsanity.com/paper/2503.19202/) (2025)
- [Zero-Shot Visualization: Exploring Text Corpora with User-Prompted Axes](https://mlsanity.com/paper/2610.06889/) (2026)

## Abstract (arXiv)

Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above mentioned challenges and address problems of supervised, zero-shot, generalized zero-shot and open set recognition using a unified framework. Specifically, we propose a weighted maximum margin framework for semantic manifold-based recognition that incorporates distance constraints from (both supervised and unsupervised) vocabulary atoms. Distance constraints ensure that labeled samples are projected closer to their correct prototypes, in the embedding space, than to others. We illustrate that resulting model shows improvements in supervised, zero-shot, generalized zero-shot, and large open set recognition, with up to 310K class vocabulary on Animal with Attributes and ImageNet datasets.