# Class Knowledge Overlay to Visual Feature Learning for Zero-Shot Image Classification

arXiv: https://arxiv.org/abs/2102.13322 · HTML: https://mlsanity.com/paper/2102.13322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 53 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Builds on (most cited first, 30 of 33)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [Evaluation of Output Embeddings for Fine-Grained Image Classification](https://mlsanity.com/paper/1409.8403/) (2014)
- [Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs](https://mlsanity.com/paper/1803.08035/) (2018)
- [Learning a Deep Embedding Model for Zero-Shot Learning](https://mlsanity.com/paper/1611.05088/) (2016)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)
- [Latent Embeddings for Zero-shot Classification](https://mlsanity.com/paper/1603.08895/) (2016)
- [Generalized Zero-Shot Learning via Synthesized Examples](https://mlsanity.com/paper/1712.03878/) (2017)
- [Synthesized Classifiers for Zero-Shot Learning](https://mlsanity.com/paper/1603.00550/) (2016)
- [A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts](https://mlsanity.com/paper/1712.01381/) (2017)
- [Zero-Shot Learning via Semantic Similarity Embedding](https://mlsanity.com/paper/1509.04767/) (2015)
- [The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos](https://mlsanity.com/paper/1812.05538/) (2018)
- [Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks](https://mlsanity.com/paper/1712.01928/) (2017)
- [Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions](https://mlsanity.com/paper/1506.00511/) (2015)
- [Generative Dual Adversarial Network for Generalized Zero-shot Learning](https://mlsanity.com/paper/1811.04857/) (2018)
- [Transductive Multi-view Zero-Shot Learning](https://mlsanity.com/paper/1501.04560/) (2015)
- [Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precision](https://mlsanity.com/paper/1709.01148/) (2017)
- [Ridge Regression, Hubness, and Zero-Shot Learning](https://mlsanity.com/paper/1507.00825/) (2015)
- [Less is more: zero-shot learning from online textual documents with noise suppression](https://mlsanity.com/paper/1604.01146/) (2016)
- [A Unified Perspective on Multi-Domain and Multi-Task Learning](https://mlsanity.com/paper/1412.7489/) (2014)
- [Recent Advances in Zero-shot Recognition](https://mlsanity.com/paper/1710.04837/) (2017)
- [Creativity Inspired Zero-Shot Learning](https://mlsanity.com/paper/1904.01109/) (2019)
- [CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language](https://mlsanity.com/paper/1909.09822/) (2019)
- [Multi-Cue Zero-Shot Learning with Strong Supervision](https://mlsanity.com/paper/1603.08754/) (2016)
- [Zero-Shot Learning via Class-Conditioned Deep Generative Models](https://mlsanity.com/paper/1711.05820/) (2017)
- [A Unified Semantic Embedding: Relating Taxonomies and Attributes](https://mlsanity.com/paper/1411.5879/) (2014)

## Abstract (arXiv)

New categories can be discovered by transforming semantic features into synthesized visual features without corresponding training samples in zero-shot image classification. Although significant progress has been made in generating high-quality synthesized visual features using generative adversarial networks, guaranteeing semantic consistency between the semantic features and visual features remains very challenging. In this paper, we propose a novel zero-shot learning approach, GAN-CST, based on class knowledge to visual feature learning to tackle the problem. The approach consists of three parts, class knowledge overlay, semi-supervised learning and triplet loss. It applies class knowledge overlay (CKO) to obtain knowledge not only from the corresponding class but also from other classes that have the knowledge overlay. It ensures that the knowledge-to-visual learning process has adequate information to generate synthesized visual features. The approach also applies a semi-supervised learning process to re-train knowledge-to-visual model. It contributes to reinforcing synthesized visual features generation as well as new category prediction. We tabulate results on a number of benchmark datasets demonstrating that the proposed model delivers superior performance over state-of-the-art approaches.