# Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes

arXiv: https://arxiv.org/abs/1703.07519 · HTML: https://mlsanity.com/paper/1703.07519/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2017.

## Built on by (most cited first, 4 of 4)

- [Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition](https://mlsanity.com/paper/2108.04603/) (2021)
- [Disentangling Semantic-to-visual Confusion for Zero-shot Learning](https://mlsanity.com/paper/2106.08605/) (2021)
- [Cross Knowledge-based Generative Zero-Shot Learning Approach with Taxonomy Regularization](https://mlsanity.com/paper/2101.09892/) (2021)
- [Multi-Knowledge Fusion for New Feature Generation in Generalized Zero-Shot Learning](https://mlsanity.com/paper/2102.11566/) (2021)

## Abstract (arXiv)

In this paper, we present a label transfer model from texts to images for image classification tasks. The problem of image classification is often much more challenging than text classification. On one hand, labeled text data is more widely available than the labeled images for classification tasks. On the other hand, text data tends to have natural semantic interpretability, and they are often more directly related to class labels. On the contrary, the image features are not directly related to concepts inherent in class labels. One of our goals in this paper is to develop a model for revealing the functional relationships between text and image features as to directly transfer intermodal and intramodal labels to annotate the images. This is implemented by learning a transfer function as a bridge to propagate the labels between two multimodal spaces. However, the intermodal label transfers could be undermined by blindly transferring the labels of noisy texts to annotate images. To mitigate this problem, we present an intramodal label transfer process, which complements the intermodal label transfer by transferring the image labels instead when relevant text is absent from the source corpus. In addition, we generalize the inter-modal label transfer to zero-shot learning scenario where there are only text examples available to label unseen classes of images without any positive image examples. We evaluate our algorithm on an image classification task and show the effectiveness with respect to the other compared algorithms.