Rubric-as-Experts: Case-Specific MQM Rubrics for Translation Error Span Detection
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have shown potential for reference-free span-level translation quality estimation (QE), yet existing approaches based on Multidimensional Quality Metrics (MQM) typically rely on fixed rubric configurations shared across translation instances. However, translation instances often differ substantially in error complexity, ambiguity, and required evaluation granularity, making static rubric allocation suboptimal for span-level error detection. We find that larger MQM subtype spaces improve error coverage but also introduce more false positives, while different translation instances prefer different rubric granularities, suggesting that evaluation spaces should be allocated dynamically for each case. Therefore, we propose a case-specific dynamic rubric framework that adaptively constructs MQM evaluation spaces for individual translation instances. Unlike methods that generate fully free-form rubrics, our framework remains grounded in the predefined MQM taxonomy while dynamically selecting suitable subtype spaces and evaluation granularity for different cases. Experiments on span-level QE benchmarks from the Conference on Machine Translation (WMT) across multiple model scales demonstrate that the proposed framework consistently improves F1 and Matthews Correlation Coefficient (MCC).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 50 references that are in the map. First posted on arXiv 19 Jun 2026.
Builds on
11 of 50 references.
References · 11
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
- 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
- 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
- 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
- 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 146
- 2025SedarEval: Automated Evaluation using Self-Adaptive Rubricscited by 19
- 2026Learning to Judge: LLMs Designing and Applying Evaluation Rubricscited by 19
- 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimationcited by 4
Built on by
Not cited yet.