# The Multi-Range Theory of Translation Quality Measurement: MQM scoring models and Statistical Quality Control

arXiv: https://arxiv.org/abs/2405.16969 · HTML: https://mlsanity.com/paper/2405.16969/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation](https://mlsanity.com/paper/2112.13833/) (2021)
- [Measuring Uncertainty in Translation Quality Evaluation (TQE)](https://mlsanity.com/paper/2111.07699/) (2021)
- [Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learning](https://mlsanity.com/paper/2312.07250/) (2023)
- [Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date overview](https://mlsanity.com/paper/1605.04515/) (2016)
- [Student's t-Distribution: On Measuring the Inter-Rater Reliability When the Observations are Scarce](https://mlsanity.com/paper/2303.04526/) (2023)
- [MTUncertainty: Assessing the Need for Post-editing of Machine Translation Outputs by Fine-tuning OpenAI LLMs](https://mlsanity.com/paper/2308.00158/) (2023)

## Built on by (most cited first, 9 of 9)

- [Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation](https://mlsanity.com/paper/2507.20301/) (2025)
- [Non-Linear Scoring Model for Translation Quality Evaluation](https://mlsanity.com/paper/2511.13467/) (2025)
- [Round-Trip Translation Reveals What Frontier Multilingual Benchmarks Miss](https://mlsanity.com/paper/2604.12911/) (2026)
- [MetaHOPE: A Metaphor-Oriented Evaluation Framework for Analysing MT and LLM Translation Errors](https://mlsanity.com/paper/2607.00848/) (2026)
- [Last Translation Benchmark](https://mlsanity.com/paper/2609.04173/) (2026)
- [Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translation](https://mlsanity.com/paper/2609.14963/) (2026)
- [EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting](https://mlsanity.com/paper/2609.08171/) (2026)
- [Lossy Compressive Text Autoencoders](https://mlsanity.com/paper/2610.10738/) (2026)
- [SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation](https://mlsanity.com/paper/2610.16646/) (2026)

## Abstract (arXiv)

The year 2024 marks the 10th anniversary of the Multidimensional Quality Metrics (MQM) framework for analytic translation quality evaluation. The MQM error typology has been widely used by practitioners in the translation and localization industry and has served as the basis for many derivative projects. The annual Conference on Machine Translation (WMT) shared tasks on both human and automatic translation quality evaluations used the MQM error typology. The metric stands on two pillars: error typology and the scoring model. The scoring model calculates the quality score from annotation data, detailing how to convert error type and severity counts into numeric scores to determine if the content meets specifications. Previously, only the raw scoring model had been published. This April, the MQM Council published the Linear Calibrated Scoring Model, officially presented herein, along with the Non-Linear Scoring Model, which had not been published before. This paper details the latest MQM developments and presents a universal approach to translation quality measurement across three sample size ranges. It also explains why Statistical Quality Control should be used for very small sample sizes, starting from a single sentence.