# Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions

arXiv: https://arxiv.org/abs/1506.00511 · HTML: https://mlsanity.com/paper/1506.00511/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2015.

## Built on by (most cited first, 30 of 64)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Task Residual for Tuning Vision-Language Models](https://mlsanity.com/paper/2211.10277/) (2022)
- [When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data](https://mlsanity.com/paper/2102.02201/) (2021)
- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [Mixed Differential Privacy in Computer Vision](https://mlsanity.com/paper/2203.11481/) (2022)
- [Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2108.06536/) (2021)
- [Multitask Vision-Language Prompt Tuning](https://mlsanity.com/paper/2211.11720/) (2022)
- [Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Prompts](https://mlsanity.com/paper/2307.11661/) (2023)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [Knowledge-aware Zero-Shot Learning: Survey and Perspective](https://mlsanity.com/paper/2103.00070/) (2021)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [On Guiding Visual Attention with Language Specification](https://mlsanity.com/paper/2202.08926/) (2022)
- [I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification](https://mlsanity.com/paper/2209.10304/) (2022)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [Semantics-Guided Contrastive Network for Zero-Shot Object detection](https://mlsanity.com/paper/2109.06062/) (2021)
- [Dual Progressive Prototype Network for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2111.02073/) (2021)
- [Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP](https://mlsanity.com/paper/2310.00927/) (2023)
- [Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2303.15322/) (2023)
- [Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models](https://mlsanity.com/paper/2207.07646/) (2022)
- [Domain-Controlled Prompt Learning](https://mlsanity.com/paper/2310.07730/) (2023)
- [Fine-Grained Zero-Shot Learning with DNA as Side Information](https://mlsanity.com/paper/2109.14133/) (2021)
- [Learning to Adapt CLIP for Few-Shot Monocular Depth Estimation](https://mlsanity.com/paper/2311.01034/) (2023)
- [EPCL: Frozen CLIP Transformer is An Efficient Point Cloud Encoder](https://mlsanity.com/paper/2212.04098/) (2022)
- [Learning Robust Visual-semantic Mapping for Zero-shot Learning](https://mlsanity.com/paper/2104.05668/) (2021)
- [Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learning](https://mlsanity.com/paper/2308.04828/) (2023)
- [Towards Zero-shot Sign Language Recognition](https://mlsanity.com/paper/2201.05914/) (2022)
- [Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models](https://mlsanity.com/paper/2311.17091/) (2023)
- [Cross-Modal Concept Learning and Inference for Vision-Language Models](https://mlsanity.com/paper/2307.15460/) (2023)

## Abstract (arXiv)

One of the main challenges in Zero-Shot Learning of visual categories is gathering semantic attributes to accompany images. Recent work has shown that learning from textual descriptions, such as Wikipedia articles, avoids the problem of having to explicitly define these attributes. We present a new model that can classify unseen categories from their textual description. Specifically, we use text features to predict the output weights of both the convolutional and the fully connected layers in a deep convolutional neural network (CNN). We take advantage of the architecture of CNNs and learn features at different layers, rather than just learning an embedding space for both modalities, as is common with existing approaches. The proposed model also allows us to automatically generate a list of pseudo- attributes for each visual category consisting of words from Wikipedia articles. We train our models end-to-end us- ing the Caltech-UCSD bird and flower datasets and evaluate both ROC and Precision-Recall curves. Our empirical results show that the proposed model significantly outperforms previous methods.