# Classifier Crafting: Turn Your ConvNet into a Zero-Shot Learner!

arXiv: https://arxiv.org/abs/2103.11112 · HTML: https://mlsanity.com/paper/2103.11112/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 20 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning](https://mlsanity.com/paper/1903.10132/) (2019)
- [Attribute Prototype Network for Zero-Shot Learning](https://mlsanity.com/paper/2008.08290/) (2020)
- [An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild](https://mlsanity.com/paper/1605.04253/) (2016)
- [Leveraging the Invariant Side of Generative Zero-Shot Learning](https://mlsanity.com/paper/1904.04092/) (2019)
- [Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications](https://mlsanity.com/paper/2003.01455/) (2020)
- [Generative Dual Adversarial Network for Generalized Zero-shot Learning](https://mlsanity.com/paper/1811.04857/) (2018)
- [Generalized Zero-Shot Learning Via Over-Complete Distribution](https://mlsanity.com/paper/2004.00666/) (2020)
- [Invertible Zero-Shot Recognition Flows](https://mlsanity.com/paper/2007.04873/) (2020)
- [Fix your classifier: the marginal value of training the last weight layer](https://mlsanity.com/paper/1801.04540/) (2018)
- [Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective](https://mlsanity.com/paper/1909.05995/) (2019)
- [Class-incremental Learning with Pre-allocated Fixed Classifiers](https://mlsanity.com/paper/2010.08657/) (2020)
- [Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches](https://mlsanity.com/paper/2007.13314/) (2020)
- [Progressive Ensemble Networks for Zero-Shot Recognition](https://mlsanity.com/paper/1805.07473/) (2018)
- [Zero-Shot Learning from Adversarial Feature Residual to Compact Visual Feature](https://mlsanity.com/paper/2008.12962/) (2020)
- [Classifier and Exemplar Synthesis for Zero-Shot Learning](https://mlsanity.com/paper/1812.06423/) (2018)
- [Multi-modal Ensemble Classification for Generalized Zero Shot Learning](https://mlsanity.com/paper/1901.04623/) (2019)

## Abstract (arXiv)

In Zero-shot learning (ZSL), we classify unseen categories using textual descriptions about their expected appearance when observed (class embeddings) and a disjoint pool of seen classes, for which annotated visual data are accessible. We tackle ZSL by casting a "vanilla" convolutional neural network (e.g. AlexNet, ResNet-101, DenseNet-201 or DarkNet-53) into a zero-shot learner. We do so by crafting the softmax classifier: we freeze its weights using fixed seen classification rules, either semantic (seen class embeddings) or visual (seen class prototypes). Then, we learn a data-driven and ZSL-tailored feature representation on seen classes only to match these fixed classification rules. Given that the latter seamlessly generalize towards unseen classes, while requiring not actual unseen data to be computed, we can perform ZSL inference by augmenting the pool of classification rules at test time while keeping the very same representation we learnt: nowhere re-training or fine-tuning on unseen data is performed. The combination of semantic and visual crafting (by simply averaging softmax scores) improves prior state-of-the-art methods in benchmark datasets for standard, inductive ZSL. After rebalancing predictions to better handle the joint inference over seen and unseen classes, we outperform prior generalized, inductive ZSL methods as well. Also, we gain interpretability at no additional cost, by using neural attention methods (e.g., grad-CAM) as they are. Code will be made publicly available.