mlsanityData to 7 Oct 2026

Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort

arXiv 2105.13385 ↗First posted 27 May 2021cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

In Machine Translation, assessing the quality of a large amount of automatic translations can be challenging. Automatic metrics are not reliable when it comes to high performing systems. In addition, resorting to human evaluators can be expensive, especially when evaluating multiple systems. To overcome the latter challenge, we propose a novel application of online learning that, given an ensemble of Machine Translation systems, dynamically converges to the best systems, by taking advantage of the human feedback available. Our experiments on WMT'19 datasets show that our online approach quickly converges to the top-3 ranked systems for the language pairs considered, despite the lack of human feedback for many translations.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 27 May 2021.

Builds on

15 of 40 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  3. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  4. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  5. 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 155
  6. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  7. 2019Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translationcited by 56
  8. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  9. 2019The University of Edinburgh's Submissions to the WMT19 News Translation Taskcited by 11
  10. 2019Microsoft Research Asia's Systems for WMT19cited by 10
  11. 2018Online Learning for Effort Reduction in Interactive Neural Machine Translationcited by 9
  12. 2018A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translationcited by 8
3 more
  1. 2017A Shared Task on Bandit Learning for Machine Translationcited by 6
  2. 2017A User-Study on Online Adaptation of Neural Machine Translation to Human Post-Editscited by 5
  3. 2020Machine Translation System Selection from Bandit Feedbackcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 1; 2024: 1; 2025: 1.

7 papers.

  1. 2025How to Select Datapoints for Efficient Human Evaluation of NLG Models?cited by 23
  2. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14
  3. 2022Toward More Effective Human Evaluation for Machine Translationcited by 10
  4. 2023Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answeringcited by 6
  5. 2022Onception: Active Learning with Expert Advice for Real World Machine Translationcited by 5
  6. 2024Better than Random: Reliable NLG Human Evaluation with Constrained Active Samplingcited by 4
  7. 2022Searching for a higher power in the human evaluation of MTcited by 3