mlsanityData to 7 Oct 2026

Preliminary Ranking of WMT25 General Machine Translation Systems

arXiv 2508.14909 ↗First posted 11 Aug 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present the preliminary rankings of machine translation (MT) systems submitted to the WMT25 General Machine Translation Shared Task, as determined by automatic evaluation metrics. Because these rankings are derived from automatic evaluation, they may exhibit a bias toward systems that employ re-ranking techniques, such as Quality Estimation or Minimum Bayes Risk decoding. The official WMT25 ranking will be based on human evaluation, which is more reliable and will supersede these results. The official WMT25 ranking will be based on human evaluation, which is more reliable and will supersede these results. The purpose of releasing these findings now is to assist task participants with their system description papers; not to provide final findings.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

13 of 21 references.

References · 13

  1. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  2. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  3. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  4. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
  5. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  6. 2024MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Taskcited by 92
  7. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  8. 2024Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translationcited by 62
  9. 2022DEMETR: Diagnosing Evaluation Metrics for Translationcited by 36
  10. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  11. 2024Machine Translation Meta Evaluation through Translation Accuracy Challenge Setscited by 18
  12. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 15
1 more
  1. 2024How Good is Zero-Shot MT Evaluation for Low Resource Indian Languages?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translationcited by 0