# Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning

arXiv: https://arxiv.org/abs/1510.03519 · HTML: https://mlsanity.com/paper/1510.03519/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2015.

## Built on by (most cited first, 18 of 18)

- [Visually Grounded Reasoning across Languages and Cultures](https://mlsanity.com/paper/2109.13238/) (2021)
- [UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training](https://mlsanity.com/paper/2104.00332/) (2021)
- [jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images](https://mlsanity.com/paper/2412.08802/) (2024)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [NLLB-CLIP -- train performant multilingual image retrieval model on a budget](https://mlsanity.com/paper/2309.01859/) (2023)
- [No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages](https://mlsanity.com/paper/2411.03769/) (2024)
- [FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learning](https://mlsanity.com/paper/2307.13214/) (2023)
- [Object-Centric Unsupervised Image Captioning](https://mlsanity.com/paper/2112.00969/) (2021)
- [CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation](https://mlsanity.com/paper/2407.01081/) (2024)
- [Towards Zero-shot Cross-lingual Image Retrieval and Tagging](https://mlsanity.com/paper/2109.07622/) (2021)
- [xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation](https://mlsanity.com/paper/2503.09313/) (2025)
- [Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunities](https://mlsanity.com/paper/2210.16960/) (2022)
- [Multilingual Vision-Language Models, A Survey](https://mlsanity.com/paper/2509.22123/) (2025)
- [Document Classification Pattern Recognition via Information Fusion: A Systematic Review of Multimodal and Multiview Representation Approaches](https://mlsanity.com/paper/2605.23910/) (2026)
- [Accessible Instruction-Following Agent](https://mlsanity.com/paper/2305.06358/) (2023)
- [Image captioning in different languages](https://mlsanity.com/paper/2407.09495/) (2024)
- [Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text](https://mlsanity.com/paper/2601.10096/) (2026)

## Abstract (arXiv)

Recently there has been a lot of interest in learning common representations for multiple views of data. Typically, such common representations are learned using a parallel corpus between the two views (say, 1M images and their English captions). In this work, we address a real-world scenario where no direct parallel data is available between two views of interest (say, V1 and V2) but parallel data is available between each of these views and a pivot view (V3). We propose a model for learning a common representation for V1, V2 and V3 using only the parallel data available between V1V3 and V2V3. The proposed model is generic and even works when there are n views of interest and only one pivot view which acts as a bridge between them. There are two specific downstream applications that we focus on (i) transfer learning between languages L1,L2,...,Ln using a pivot language L and (ii) cross modal access between images and a language L1 using a pivot language L2. Our model achieves state-of-the-art performance in multilingual document classification on the publicly available multilingual TED corpus and promising results in multilingual multimodal retrieval on a new dataset created and released as a part of this work.