Long-context Reference-based MT Quality Estimation
Not on the map. Why?
Abstract (arXiv)
In this paper, we present our submission to the Tenth Conference on Machine Translation (WMT25) Shared Task on Automated Translation Quality Evaluation. Our systems are built upon the COMET framework and trained to predict segment-level Error Span Annotation (ESA) scores using augmented long-context data. To construct long-context training data, we concatenate in-domain, human-annotated sentences and compute a weighted average of their scores. We integrate multiple human judgment datasets (MQM, SQM, and DA) by normalising their scales and train multilingual regression models to predict quality scores from the source, hypothesis, and reference translations. Experimental results show that incorporating long-context information improves correlations with human judgments compared to models trained only on short segments.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 17 Sep 2025.
Builds on
18 of 39 references.
References · 18
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
- 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
- 2017Evaluating Discourse Phenomena in Neural Machine Translationcited by 87
- 2021Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Paperscited by 71
- 2024Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translationcited by 62
6 more
- 2018Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translationcited by 58
- 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
- 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
- 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
- 2020Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehensioncited by 29
- 2023SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Windowcited by 11
Built on by
Citing papers per year: 2025: 2.
2 papers.