# BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation

arXiv: https://arxiv.org/abs/2502.04314 · HTML: https://mlsanity.com/paper/2502.04314/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.16039/) (2023)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [EuroLLM: Multilingual Language Models for Europe](https://mlsanity.com/paper/2409.16235/) (2024)
- [MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://mlsanity.com/paper/2410.03983/) (2024)
- [A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics](https://mlsanity.com/paper/1812.08092/) (2018)
- [CEFR-Based Sentence Difficulty Annotation and Assessment](https://mlsanity.com/paper/2210.11766/) (2022)
- [Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources](https://mlsanity.com/paper/2211.15649/) (2022)
- [MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognition](https://mlsanity.com/paper/2208.14536/) (2022)
- [MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector](https://mlsanity.com/paper/2401.05060/) (2024)
- [GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding](https://mlsanity.com/paper/2406.09781/) (2024)
- [Methods for Quantifying Dataset Similarity: a Review, Taxonomy and Comparison](https://mlsanity.com/paper/2312.04078/) (2023)
- [COMET-QE and Active Learning for Low-Resource Machine Translation](https://mlsanity.com/paper/2210.15696/) (2022)
- [2M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Dataset](https://mlsanity.com/paper/2412.08274/) (2024)

## Built on by (most cited first, 22 of 22)

- [Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models](https://mlsanity.com/paper/2602.11961/) (2026)
- [Translation Asymmetry in LLMs as a Data Augmentation Factor: A Case Study for 6 Romansh Language Varieties](https://mlsanity.com/paper/2603.25489/) (2026)
- [RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment](https://mlsanity.com/paper/2604.22520/) (2026)
- [OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents](https://mlsanity.com/paper/2608.08775/) (2026)
- [ChiKhaPo: A Large-Scale Multilingual Benchmark for Evaluating Lexical Comprehension and Generation in Large Language Models](https://mlsanity.com/paper/2510.16928/) (2025)
- [HEAD-QA v2: Expanding a Healthcare Benchmark for Reasoning](https://mlsanity.com/paper/2511.15355/) (2025)
- [Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent](https://mlsanity.com/paper/2606.03259/) (2026)
- [OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languages](https://mlsanity.com/paper/2606.09553/) (2026)
- [ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs](https://mlsanity.com/paper/2607.00171/) (2026)
- [Cultivar: A Contrastive and Locale-Oriented Translation Benchmark for Investigating Contamination and Localisation Robustness](https://mlsanity.com/paper/2608.09766/) (2026)
- [Last Translation Benchmark](https://mlsanity.com/paper/2609.04173/) (2026)
- [ReBeCA: Unveiling Interpretable Behavior Hierarchy behind the Iterative Self-Reflection of Language Models with Causal Analysis](https://mlsanity.com/paper/2602.06373/) (2026)
- [Dialects of Translationese Shape Language Model Learning](https://mlsanity.com/paper/2602.16469/) (2026)
- [Model-Based Quality Assessment for Massively Multilingual Parallel Data](https://mlsanity.com/paper/2606.00285/) (2026)
- [Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation](https://mlsanity.com/paper/2606.18389/) (2026)
- [There is No Theoretical Curse of Multilinguality For Embedding Space Structure](https://mlsanity.com/paper/2608.17088/) (2026)
- [Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?](https://mlsanity.com/paper/2608.27115/) (2026)
- [The University of Melbourne WMT 2026 CreoleMT Submission: A Domain-Balanced Approach to Low-Resource Pacific Creole Machine Translation](https://mlsanity.com/paper/2609.13615/) (2026)
- [ReMova: Fine-tuning LLMs for English to Belarusian translation](https://mlsanity.com/paper/2609.16427/) (2026)
- [LocQE: Principled Domain Adaptation for Localisation Quality Estimation by Leveraging Post-Edits](https://mlsanity.com/paper/2609.18720/) (2026)
- [Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Models](https://mlsanity.com/paper/2610.01082/) (2026)
- [Swiss4Arg: A Parallel Dataset for Multilingual Argument Mining across Swiss National Languages](https://mlsanity.com/paper/2610.17894/) (2026)

## Abstract (arXiv)

BOUQuET is a multi-way, multicentric and multi-register/domain dataset and benchmark, and a broader collaborative initiative. This dataset is handcrafted in 8 non-English languages. Each of these source languages are representative of the most widely spoken ones and therefore they have the potential to serve as pivot languages that will enable more accurate translations. The dataset is multicentric to enforce representation of multilingual language features. In addition, the dataset goes beyond the sentence level, as it is organized in paragraphs of various lengths. Compared with related machine translation datasets, we show that BOUQuET has a broader representation of domains while simplifying the translation task for non-experts. Therefore, BOUQuET is specially suitable for crowd-source extension for which we are launching a call aiming at collecting a multi-way parallel corpus covering any written language.