# Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings

arXiv: https://arxiv.org/abs/1811.01136 · HTML: https://mlsanity.com/paper/1811.01136/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2018.

## Built on by (most cited first, 30 of 83)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Gender Bias in Masked Language Models for Multiple Languages](https://mlsanity.com/paper/2205.00551/) (2022)
- [Democratizing Neural Machine Translation with OPUS-MT](https://mlsanity.com/paper/2212.01936/) (2022)
- [Facebook AI WMT21 News Translation Task Submission](https://mlsanity.com/paper/2108.03265/) (2021)
- [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](https://mlsanity.com/paper/2205.12654/) (2022)
- [LongForm: Effective Instruction Tuning with Reverse Instructions](https://mlsanity.com/paper/2304.08460/) (2023)
- [CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs](https://mlsanity.com/paper/2112.08804/) (2021)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs](https://mlsanity.com/paper/2502.14830/) (2025)
- [Discovering Language-neutral Sub-networks in Multilingual Language Models](https://mlsanity.com/paper/2205.12672/) (2022)
- [Neural Data-to-Text Generation with LM-based Text Augmentation](https://mlsanity.com/paper/2102.03556/) (2021)
- [Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment](https://mlsanity.com/paper/2101.00148/) (2021)
- [How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language](https://mlsanity.com/paper/2204.11909/) (2022)
- [Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training](https://mlsanity.com/paper/2205.10471/) (2022)
- [USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2202.10062/) (2022)
- [Krikri: Advancing Open Large Language Models for Greek](https://mlsanity.com/paper/2505.13772/) (2025)
- [Phrase-level Active Learning for Neural Machine Translation](https://mlsanity.com/paper/2106.11375/) (2021)
- [Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corpora](https://mlsanity.com/paper/2402.07446/) (2024)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Multilingual Representation Distillation with Contrastive Learning](https://mlsanity.com/paper/2210.05033/) (2022)
- [LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation](https://mlsanity.com/paper/2302.08387/) (2023)
- [Locating Information Gaps and Narrative Inconsistencies Across Languages: A Case Study of LGBT People Portrayals on Wikipedia](https://mlsanity.com/paper/2410.04282/) (2024)
- [Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining](https://mlsanity.com/paper/2105.10419/) (2021)
- [Dual-Alignment Pre-training for Cross-lingual Sentence Embedding](https://mlsanity.com/paper/2305.09148/) (2023)
- [Assessing Reference-Free Peer Evaluation for Machine Translation](https://mlsanity.com/paper/2104.05146/) (2021)
- [A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space](https://mlsanity.com/paper/2109.06324/) (2021)
- [Language Agnostic Multilingual Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2210.06633/) (2022)
- [Aligning Cross-lingual Sentence Representations with Dual Momentum Contrast](https://mlsanity.com/paper/2109.00253/) (2021)

## Abstract (arXiv)

Machine translation is highly sensitive to the size and quality of the training data, which has led to an increasing interest in collecting and filtering large parallel corpora. In this paper, we propose a new method for this task based on multilingual sentence embeddings. In contrast to previous approaches, which rely on nearest neighbor retrieval with a hard threshold over cosine similarity, our proposed method accounts for the scale inconsistencies of this measure, considering the margin between a given sentence pair and its closest candidates instead. Our experiments show large improvements over existing methods. We outperform the best published results on the BUCC mining task and the UN reconstruction task by more than 10 F1 and 30 precision points, respectively. Filtering the English-German ParaCrawl corpus with our approach, we obtain 31.2 BLEU points on newstest2014, an improvement of more than one point over the best official filtered version.