# Filtering and Mining Parallel Data in a Joint Multilingual Space

arXiv: https://arxiv.org/abs/1805.09822 · HTML: https://mlsanity.com/paper/1805.09822/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2018.

## Built on by (most cited first, 30 of 32)

- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](https://mlsanity.com/paper/2205.12654/) (2022)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation](https://mlsanity.com/paper/2205.08180/) (2022)
- [Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?](https://mlsanity.com/paper/2203.08850/) (2022)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment](https://mlsanity.com/paper/2101.00148/) (2021)
- [MSCTD: A Multimodal Sentiment Chat Translation Dataset](https://mlsanity.com/paper/2202.13645/) (2022)
- [Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning](https://mlsanity.com/paper/2405.19462/) (2024)
- [Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining](https://mlsanity.com/paper/2105.10419/) (2021)
- [Dual-Alignment Pre-training for Cross-lingual Sentence Embedding](https://mlsanity.com/paper/2305.09148/) (2023)
- [Language Agnostic Multilingual Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2210.06633/) (2022)
- [Very Low Resource Sentence Alignment: Luhya and Swahili](https://mlsanity.com/paper/2211.00046/) (2022)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer](https://mlsanity.com/paper/2110.02950/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval](https://mlsanity.com/paper/2212.10726/) (2022)
- [Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation](https://mlsanity.com/paper/2306.01382/) (2023)
- [MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained Language Model for Machine Translation and Semantic Retrieval for P=ali, Sanskrit, Buddhist Chinese, and Tibetan](https://mlsanity.com/paper/2601.06400/) (2026)
- [Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages](https://mlsanity.com/paper/2103.13272/) (2021)
- [BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation](https://mlsanity.com/paper/2111.06787/) (2021)
- [OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval](https://mlsanity.com/paper/2205.08605/) (2022)
- [Secoco: Self-Correcting Encoding for Neural Machine Translation](https://mlsanity.com/paper/2108.12137/) (2021)
- [Cross-Lingual Phrase Retrieval](https://mlsanity.com/paper/2204.08887/) (2022)
- [Mitigating Semantic Leakage in Cross-lingual Embeddings via Orthogonality Constraint](https://mlsanity.com/paper/2409.15664/) (2024)
- [Learning How to Translate North Korean through South Korean](https://mlsanity.com/paper/2201.11258/) (2022)
- [Finetuning a Kalaallisut-English machine translation system using web-crawled data](https://mlsanity.com/paper/2206.02230/) (2022)
- [A General-Purpose Multilingual Document Encoder](https://mlsanity.com/paper/2305.07016/) (2023)

## Abstract (arXiv)

We learn a joint multilingual sentence embedding and use the distance between sentences in different languages to filter noisy parallel data and to mine for parallel data in large news collections. We are able to improve a competitive baseline on the WMT'14 English to German task by 0.3 BLEU by filtering out 25% of the training data. The same approach is used to mine additional bitexts for the WMT'14 system and to obtain competitive results on the BUCC shared task to identify parallel sentences in comparable corpora. The approach is generic, it can be applied to many language pairs and it is independent of the architecture of the machine translation system.