mlsanityData to 14 Oct 2026

Unbabel's Participation in the WMT20 Metrics Shared Task

arXiv 2010.15535 ↗First posted 29 Oct 2020cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

We present the contribution of the Unbabel team to the WMT 2020 Shared Task on Metrics. We intend to participate on the segment-level, document-level and system-level tracks on all language pairs, as well as the 'QE as a Metric' track. Accordingly, we illustrate results of our models in these tracks with reference to test sets from the previous year. Our submissions build upon the recently proposed COMET framework: We train several estimator models to regress on different human-generated quality scores and a novel ranking model trained on relative ranks obtained from Direct Assessments. We also propose a simple technique for converting segment-level predictions into a document-level score. Overall, our systems achieve strong results for all language pairs on previous test sets and in many cases set a new state-of-the-art.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 11; 2023: 18; 2024: 14; 2025: 7; 2026: 3.

56 papers.

Sort:

By area:

  1. 2021Learning Compact Metrics for MTcited by 93
  2. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  3. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  4. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  5. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  6. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  7. 2022Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMETcited by 41
  8. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  9. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  10. 2024Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignmentcited by 25
  11. 2023Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluationcited by 19
  12. 2023A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translationcited by 19
38 more
  1. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  2. 2022MENLI: Robust Evaluation Metrics from Natural Language Inferencecited by 18
  3. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  4. 2024MT-Ranker: Reference-free machine translation evaluation by inter-system rankingcited by 14
  5. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 14
  6. 2022Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translationcited by 13
  7. 2023CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translationcited by 12
  8. 2023Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Referencecited by 11
  9. 2023SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Windowcited by 11
  10. 2023Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weightingcited by 11
  11. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 11
  12. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  13. 2022COMET-QE and Active Learning for Low-Resource Machine Translationcited by 10
  14. 2024Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruningcited by 10
  15. 2024Pitfalls and Outlooks in Using COMETcited by 10
  16. 2024Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimationcited by 10
  17. 2025Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progresscited by 10
  18. 2022Disentangling Uncertainty in Machine Translation Evaluationcited by 9
  19. 2022MT Metrics Correlate with Human Ratings of Simultaneous Speech Translationcited by 9
  20. 2023Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Modelcited by 9
  21. 2021Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effortcited by 7
  22. 2023A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-Supervisioncited by 6
  23. 2023Learning Evaluation Models from Large Language Models for Sequence Generationcited by 6
  24. 2024Is Context Helpful for Chat Translation Evaluation?cited by 5
  25. 2024Recent Advances in End-to-End Simultaneous Speech Translationcited by 5
  26. 2024GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learningcited by 5
  27. 2022Twist Decoding: Diverse Generators Guide Each Othercited by 4
  28. 2023NoRefER: a Referenceless Quality Metric for Automatic Speech Recognition via Semi-Supervised Language Model Fine-Tuning with Contrastive Learningcited by 4
  29. 2024Word-Level ASR Quality Estimation for Efficient Corpus Sampling and Post-Editing through Analyzing Attentions of a Reference-Free Metriccited by 4
  30. 2023Long-form Simultaneous Speech Translation: Thesis Proposalcited by 3
  31. 2026Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Frameworkcited by 3
  32. 2023Robustness of Multi-Source MT to Transcription Errorscited by 2
  33. 2024Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptionscited by 2
  34. 2024PyMarian: Fast Neural Machine Translation and Evaluation in Pythoncited by 2
  35. 2025Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translationcited by 2
  36. 2025Long-context Reference-based MT Quality Estimationcited by 2
  37. 2024SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translationcited by 1
  38. 2025Faster Machine Translation Ensembling with Reinforcement Learning and Competitive Correctioncited by 1

50 most cited of 56 All 56 (JSON)