# Model-Based Quality Assessment for Massively Multilingual Parallel Data

arXiv: https://arxiv.org/abs/2606.00285 · HTML: https://mlsanity.com/paper/2606.00285/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 29 May 2026.

## Builds on (most cited first, 21 of 21)

- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://mlsanity.com/paper/2410.03983/) (2024)
- [Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2309.11925/) (2023)
- [Batch Prompting: Efficient Inference with Large Language Model APIs](https://mlsanity.com/paper/2301.08721/) (2023)
- [Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings](https://mlsanity.com/paper/1811.01136/) (2018)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [M-Prometheus: A Suite of Open Multilingual LLM Judges](https://mlsanity.com/paper/2504.04953/) (2025)
- [Filtering and Mining Parallel Data in a Joint Multilingual Space](https://mlsanity.com/paper/1805.09822/) (2018)
- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [There's no Data Like Better Data: Using QE Metrics for MT Data Filtering](https://mlsanity.com/paper/2311.05350/) (2023)
- [Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings](https://mlsanity.com/paper/2010.07761/) (2020)
- [From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation](https://mlsanity.com/paper/2403.14118/) (2024)
- [Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling](https://mlsanity.com/paper/2504.13630/) (2025)
- [TASER: Translation Assessment via Systematic Evaluation and Reasoning](https://mlsanity.com/paper/2510.00255/) (2025)

## Abstract (arXiv)

Large-scale multilingual bitext often contains two distinct problems: non-parallel sentence pairs and low-quality translations. We decompose model-based assessment for such data into two independent components: parallelism assessment with multilingual embeddings and reference-free quality estimation (QE). For parallelism, we benchmark four embedding models on FLORES-200 and BOUQuET retrieval tasks, covering 6,654 source--target directions in our target language-pair inventory. For QE, we evaluate nine reference-free evaluators on professional FLORES-200 translations across 41,412 ordered source--target directions. Results show that no model is universally reliable across translation directions. Naive QE ensembles dilute strong model signals, while documented target-language coverage is strongly associated with higher QE scores. Overall, these findings suggest that multilingual parallel-data assessment is best approached as a direction-aware routing and calibration problem, where no single universal metric is expected to suffice across all languages.