# Task-Independent Knowledge Makes for Transferable Representations for Generalized Zero-Shot Learning

arXiv: https://arxiv.org/abs/2104.01832 · HTML: https://mlsanity.com/paper/2104.01832/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 5 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Audio-Visual Instance Discrimination with Cross-Modal Agreement](https://mlsanity.com/paper/2004.12943/) (2020)
- [f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning](https://mlsanity.com/paper/1903.10132/) (2019)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Latent Embeddings for Zero-shot Classification](https://mlsanity.com/paper/1603.08895/) (2016)
- [Synthesized Classifiers for Zero-Shot Learning](https://mlsanity.com/paper/1603.00550/) (2016)
- [Rethinking Knowledge Graph Propagation for Zero-Shot Learning](https://mlsanity.com/paper/1805.11724/) (2018)
- [Multi-modal Cycle-consistent Generalized Zero-Shot Learning](https://mlsanity.com/paper/1808.00136/) (2018)
- [Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks](https://mlsanity.com/paper/1712.01928/) (2017)
- [Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions](https://mlsanity.com/paper/1506.00511/) (2015)
- [Semantic-Guided Multi-Attention Localization for Zero-Shot Learning](https://mlsanity.com/paper/1903.00502/) (2019)
- [Generalized Zero-Shot Learning Via Over-Complete Distribution](https://mlsanity.com/paper/2004.00666/) (2020)
- [Invertible Zero-Shot Recognition Flows](https://mlsanity.com/paper/2007.04873/) (2020)
- [Ridge Regression, Hubness, and Zero-Shot Learning](https://mlsanity.com/paper/1507.00825/) (2015)
- [Preserving Semantic Relations for Zero-Shot Learning](https://mlsanity.com/paper/1803.03049/) (2018)
- [Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition](https://mlsanity.com/paper/2006.08939/) (2020)
- [Progressive Ensemble Networks for Zero-Shot Recognition](https://mlsanity.com/paper/1805.07473/) (2018)
- [Semantically Aligned Bias Reducing Zero Shot Learning](https://mlsanity.com/paper/1904.07659/) (2019)

## Built on by (most cited first, 3 of 3)

- [Region Semantically Aligned Network for Zero-Shot Learning](https://mlsanity.com/paper/2110.07130/) (2021)
- [Cluster-based Contrastive Disentangling for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2203.02648/) (2022)
- [High-Discriminative Attribute Feature Learning for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2404.04953/) (2024)

## Abstract (arXiv)

Generalized Zero-Shot Learning (GZSL) targets recognizing new categories by learning transferable image representations. Existing methods find that, by aligning image representations with corresponding semantic labels, the semantic-aligned representations can be transferred to unseen categories. However, supervised by only seen category labels, the learned semantic knowledge is highly task-specific, which makes image representations biased towards seen categories. In this paper, we propose a novel Dual-Contrastive Embedding Network (DCEN) that simultaneously learns task-specific and task-independent knowledge via semantic alignment and instance discrimination. First, DCEN leverages task labels to cluster representations of the same semantic category by cross-modal contrastive learning and exploring semantic-visual complementarity. Besides task-specific knowledge, DCEN then introduces task-independent knowledge by attracting representations of different views of the same image and repelling representations of different images. Compared to high-level seen category supervision, this instance discrimination supervision encourages DCEN to capture low-level visual knowledge, which is less biased toward seen categories and alleviates the representation bias. Consequently, the task-specific and task-independent knowledge jointly make for transferable representations of DCEN, which obtains averaged 4.1% improvement on four public benchmarks.