# Learning Joint Multilingual Sentence Representations with Neural Machine Translation

arXiv: https://arxiv.org/abs/1704.04154 · HTML: https://mlsanity.com/paper/1704.04154/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2017.

## Built on by (most cited first, 30 of 52)

- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data](https://mlsanity.com/paper/2407.14985/) (2024)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation](https://mlsanity.com/paper/2205.08180/) (2022)
- [Multilingual Routing in Mixture-of-Experts](https://mlsanity.com/paper/2510.04694/) (2025)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [Residual Quantization with Implicit Neural Codebooks](https://mlsanity.com/paper/2401.14732/) (2024)
- [Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scale](https://mlsanity.com/paper/2305.13198/) (2023)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [EASE: Entity-Aware Contrastive Learning of Sentence Embedding](https://mlsanity.com/paper/2205.04260/) (2022)
- [There's no Data Like Better Data: Using QE Metrics for MT Data Filtering](https://mlsanity.com/paper/2311.05350/) (2023)
- [Obtaining Better Static Word Embeddings Using Contextual Embedding Models](https://mlsanity.com/paper/2106.04302/) (2021)
- [Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction](https://mlsanity.com/paper/2109.06798/) (2021)
- [BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metric](https://mlsanity.com/paper/2212.08486/) (2022)
- [L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERT](https://mlsanity.com/paper/2304.11434/) (2023)
- [Cross-lingual Transfer of Monolingual Models](https://mlsanity.com/paper/2109.07348/) (2021)
- [Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2204.06812/) (2022)
- [Preparing the Vuk'uzenzele and ZA-gov-multilingual South African multilingual corpora](https://mlsanity.com/paper/2303.03750/) (2023)
- [Cross-Lingual Transfer Learning for Speech Translation](https://mlsanity.com/paper/2407.01130/) (2024)
- [Graph-based Multilingual Product Retrieval in E-commerce Search](https://mlsanity.com/paper/2105.02978/) (2021)
- [Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining](https://mlsanity.com/paper/2105.10419/) (2021)
- [FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task](https://mlsanity.com/paper/2107.06959/) (2021)
- [NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures](https://mlsanity.com/paper/2204.13692/) (2022)
- [Very Low Resource Sentence Alignment: Luhya and Swahili](https://mlsanity.com/paper/2211.00046/) (2022)
- [Recovering document annotations for sentence-level bitext](https://mlsanity.com/paper/2406.03869/) (2024)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [Lightweight Cross-Lingual Sentence Representation Learning](https://mlsanity.com/paper/2105.13856/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval](https://mlsanity.com/paper/2212.10726/) (2022)

## Abstract (arXiv)

In this paper, we use the framework of neural machine translation to learn joint sentence representations across six very different languages. Our aim is that a representation which is independent of the language, is likely to capture the underlying semantics. We define a new cross-lingual similarity measure, compare up to 1.4M sentence representations and study the characteristics of close sentences. We provide experimental evidence that sentences that are close in embedding space are indeed semantically highly related, but often have quite different structure and syntax. These relations also hold when comparing sentences in different languages.