# Image-Text Multi-Modal Representation Learning by Adversarial Backpropagation

arXiv: https://arxiv.org/abs/1612.08354 · HTML: https://mlsanity.com/paper/1612.08354/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Dec 2016.

## Built on by (most cited first, 2 of 2)

- [Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations](https://mlsanity.com/paper/2407.04955/) (2024)
- [Self-Augmented Multi-Modal Feature Embedding](https://mlsanity.com/paper/2103.04731/) (2021)

## Abstract (arXiv)

We present novel method for image-text multi-modal representation learning. In our knowledge, this work is the first approach of applying adversarial learning concept to multi-modal learning and not exploiting image-text pair information to learn multi-modal feature. We only use category information in contrast with most previous methods using image-text pair information for multi-modal embedding. In this paper, we show that multi-modal feature can be achieved without image-text pair information and our method makes more similar distribution with image and text in multi-modal feature space than other methods which use image-text pair information. And we show our multi-modal feature has universal semantic information, even though it was trained for category prediction. Our model is end-to-end backpropagation, intuitive and easily extended to other multi-modal learning work.