# Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corpora

arXiv: https://arxiv.org/abs/2402.07446 · HTML: https://mlsanity.com/paper/2402.07446/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://mlsanity.com/paper/1911.04944/) (2019)
- [Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks](https://mlsanity.com/paper/2305.10160/) (2023)
- [CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs](https://mlsanity.com/paper/1911.06154/) (2019)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings](https://mlsanity.com/paper/1811.01136/) (2018)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World](https://mlsanity.com/paper/2210.08523/) (2022)
- [Survey on Publicly Available Sinhala Natural Language Processing Tools and Research](https://mlsanity.com/paper/1906.02358/) (2019)
- [Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?](https://mlsanity.com/paper/2203.08850/) (2022)
- [Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translation](https://mlsanity.com/paper/2011.02821/) (2020)
- [XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment](https://mlsanity.com/paper/2104.08597/) (2021)
- [Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation](https://mlsanity.com/paper/2306.01382/) (2023)

## Built on by (most cited first, 13 of 13)

- [Sinhala Transliteration: A Comparative Analysis Between Rule-based and Seq2Seq Approaches](https://mlsanity.com/paper/2501.00529/) (2024)
- [Leveraging Large Language Models to Measure Gender Representation Bias in Gendered Language Corpora](https://mlsanity.com/paper/2406.13677/) (2024)
- [How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translation](https://mlsanity.com/paper/2407.02208/) (2024)
- [How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations](https://mlsanity.com/paper/2508.05625/) (2025)
- [SiTSE: Sinhala Text Simplification Dataset and Evaluation](https://mlsanity.com/paper/2412.01293/) (2024)
- [Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics](https://mlsanity.com/paper/2502.19074/) (2025)
- [Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translation](https://mlsanity.com/paper/2412.19522/) (2024)
- [Pralekha: Cross-Lingual Document Alignment for Indic Languages](https://mlsanity.com/paper/2411.19096/) (2024)
- [ACADATA: Parallel Dataset of Academic Data for Machine Translation](https://mlsanity.com/paper/2510.12621/) (2025)
- [EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation](https://mlsanity.com/paper/2609.29511/) (2026)
- [Unsupervised Bilingual Lexicon Induction for Low Resource Languages](https://mlsanity.com/paper/2412.16894/) (2024)
- [Why Low-Resource NLP Needs More Than Cross-Lingual Transfer: Lessons Learned from Luxembourgish](https://mlsanity.com/paper/2605.10714/) (2026)
- [Trilingual Topic Modeling of Sri Lankan Parliamentary Debates](https://mlsanity.com/paper/2608.20365/) (2026)

## Abstract (arXiv)

We conducted a detailed analysis on the quality of web-mined corpora for two low-resource languages (making three language pairs, English-Sinhala, English-Tamil and Sinhala-Tamil). We ranked each corpus according to a similarity measure and carried out an intrinsic and extrinsic evaluation on different portions of this ranked corpus. We show that there are significant quality differences between different portions of web-mined corpora and that the quality varies across languages and datasets. We also show that, for some web-mined datasets, Neural Machine Translation (NMT) models trained with their highest-ranked 25k portion can be on par with human-curated datasets.