# Zero-Shot Learning Through Cross-Modal Transfer

arXiv: https://arxiv.org/abs/1301.3666 · HTML: https://mlsanity.com/paper/1301.3666/

## Key facts

- Cited by 237 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2013.

## Built on by (most cited first, 30 of 237)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Visual Classification via Description from Large Language Models](https://mlsanity.com/paper/2210.07183/) (2022)
- [Simple Open-Vocabulary Object Detection with Vision Transformers](https://mlsanity.com/paper/2205.06230/) (2022)
- [Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts](https://mlsanity.com/paper/2206.02770/) (2022)
- [Versatile Diffusion: Text, Images and Variations All in One Diffusion Model](https://mlsanity.com/paper/2211.08332/) (2022)
- [Combined Scaling for Zero-shot Transfer Learning](https://mlsanity.com/paper/2111.10050/) (2021)
- [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021)
- [DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection](https://mlsanity.com/paper/2209.09407/) (2022)
- [Align and Prompt: Video-and-Language Pre-training with Entity Prompts](https://mlsanity.com/paper/2112.09583/) (2021)
- [MultiBench: Multiscale Benchmarks for Multimodal Representation Learning](https://mlsanity.com/paper/2107.07502/) (2021)
- [Task Residual for Tuning Vision-Language Models](https://mlsanity.com/paper/2211.10277/) (2022)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [CLIPood: Generalizing CLIP to Out-of-Distributions](https://mlsanity.com/paper/2302.00864/) (2023)
- [Contrastive Embedding for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2103.16173/) (2021)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [K-LITE: Learning Transferable Visual Models with External Knowledge](https://mlsanity.com/paper/2204.09222/) (2022)
- [Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings](https://mlsanity.com/paper/2202.06671/) (2022)
- [BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP](https://mlsanity.com/paper/2311.16194/) (2023)
- [Frozen Language Model Helps ECG Zero-Shot Learning](https://mlsanity.com/paper/2303.12311/) (2023)
- [Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search](https://mlsanity.com/paper/2102.01645/) (2021)
- [Subspace Regularizers for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2110.07059/) (2021)
- [Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2108.06230/) (2021)
- [Mixed Differential Privacy in Computer Vision](https://mlsanity.com/paper/2203.11481/) (2022)
- [I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification](https://mlsanity.com/paper/2212.02291/) (2022)
- [Multitask Vision-Language Prompt Tuning](https://mlsanity.com/paper/2211.11720/) (2022)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Prompts](https://mlsanity.com/paper/2307.11661/) (2023)

## Abstract (arXiv)

This work introduces a model that can recognize objects in images even if no training data is available for the objects. The only necessary knowledge about the unseen categories comes from unsupervised large text corpora. In our zero-shot framework distributional information in language can be seen as spanning a semantic basis for understanding what objects look like. Most previous zero-shot learning models can only differentiate between unseen classes. In contrast, our model can both obtain state of the art performance on classes that have thousands of training images and obtain reasonable performance on unseen classes. This is achieved by first using outlier detection in the semantic space and then two separate recognition models. Furthermore, our model does not require any manually defined semantic features for either words or images.