# Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model

arXiv: https://arxiv.org/abs/1810.12836 · HTML: https://mlsanity.com/paper/1810.12836/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 30 of 46)

- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [Multilingual and Cross-Lingual Intent Detection from Spoken Data](https://mlsanity.com/paper/2104.08524/) (2021)
- [Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval](https://mlsanity.com/paper/2103.11920/) (2021)
- [Are Pre-trained Convolutions Better than Pre-trained Transformers?](https://mlsanity.com/paper/2105.03322/) (2021)
- [English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings](https://mlsanity.com/paper/2211.06127/) (2022)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors](https://mlsanity.com/paper/2110.04399/) (2021)
- [On Cross-Lingual Retrieval with Multilingual Text Encoders](https://mlsanity.com/paper/2112.11031/) (2021)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations](https://mlsanity.com/paper/2109.04727/) (2021)
- [Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula](https://mlsanity.com/paper/2212.01375/) (2022)
- [Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval](https://mlsanity.com/paper/2101.08370/) (2021)
- [Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model](https://mlsanity.com/paper/2111.09634/) (2021)
- [Detecting Text Formality: A Study of Text Classification Approaches](https://mlsanity.com/paper/2204.08975/) (2022)
- [Language Models are Universal Embedders](https://mlsanity.com/paper/2310.08232/) (2023)
- [A Dataset and Baselines for Multilingual Reply Suggestion](https://mlsanity.com/paper/2106.02017/) (2021)
- [Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations](https://mlsanity.com/paper/2109.13087/) (2021)
- [Dual-Alignment Pre-training for Cross-lingual Sentence Embedding](https://mlsanity.com/paper/2305.09148/) (2023)
- [LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings](https://mlsanity.com/paper/2412.03331/) (2024)
- [NewsEmbed: Modeling News through Pre-trained Document Representations](https://mlsanity.com/paper/2106.00590/) (2021)
- [Learning Federated Representations and Recommendations with Limited Negatives](https://mlsanity.com/paper/2108.07931/) (2021)
- [Lightweight Cross-Lingual Sentence Representation Learning](https://mlsanity.com/paper/2105.13856/) (2021)
- [PAUSE: Positive and Annealed Unlabeled Sentence Embedding](https://mlsanity.com/paper/2109.03155/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Towards Zero-shot Cross-lingual Image Retrieval and Tagging](https://mlsanity.com/paper/2109.07622/) (2021)
- [Improving Multi-lingual Alignment Through Soft Contrastive Learning](https://mlsanity.com/paper/2405.16155/) (2024)
- [A Neighbourhood Framework for Resource-Lean Content Flagging](https://mlsanity.com/paper/2103.17055/) (2021)
- [Deception detection in text and its relation to the cultural dimension of individualism/collectivism](https://mlsanity.com/paper/2105.12530/) (2021)
- [Multi-task Learning for Cross-Lingual Sentiment Analysis](https://mlsanity.com/paper/2212.07160/) (2022)
- [Cross-Lingual Phrase Retrieval](https://mlsanity.com/paper/2204.08887/) (2022)

## Abstract (arXiv)

A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual text representations that can be used to transfer the performance from training on English tasks to non-English tasks, despite little to no task-specific non-English data. In this paper, we explore a natural setup for learning cross-lingual sentence representations: the dual-encoder. We provide a comprehensive evaluation of our cross-lingual representations on a number of monolingual, cross-lingual, and zero-shot/few-shot learning tasks, and also give an analysis of different learned cross-lingual embedding spaces.