# Multitask Text-to-Visual Embedding with Titles and Clickthrough Data

arXiv: https://arxiv.org/abs/1905.13339 · HTML: https://mlsanity.com/paper/1905.13339/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 1 of 1)

- [Towards Zero-shot Cross-lingual Image Retrieval and Tagging](https://mlsanity.com/paper/2109.07622/) (2021)

## Abstract (arXiv)

Text-visual (or called semantic-visual) embedding is a central problem in vision-language research. It typically involves mapping of an image and a text description to a common feature space through a CNN image encoder and a RNN language encoder. In this paper, we propose a new method for learning text-visual embedding using both image titles and click-through data from an image search engine. We also propose a new triplet loss function by modeling positive awareness of the embedding, and introduce a novel mini-batch-based hard negative sampling approach for better data efficiency in the learning process. Experimental results show that our proposed method outperforms existing methods, and is also effective for real-world text-to-visual retrieval.