mlsanityData to 14 Oct 2026

Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translation

arXiv 2503.08327 ↗First posted 11 Mar 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

As automatic metrics become increasingly stronger and widely adopted, the risk of unintentionally "gaming the metric" during model development rises. This issue is caused by metric interference (MINT), i.e., the use of the same or related metrics for both model tuning and evaluation. MINT can misguide practitioners into being overoptimistic about the performance of their systems: as system outputs become a function of the interfering metric, their estimated quality loses correlation with human judgments. In this work, we analyze two common cases of MINT in machine translation-related tasks: filtering of training data, and decoding with quality signals. Importantly, we find that MINT strongly distorts instance-level metric scores, even when metrics are not directly optimized for-questioning the common strategy of leveraging a different, yet related metric for evaluation that is not used for tuning. To address this problem, we propose MINTADJUST, a method for more reliable evaluation under MINT. On the WMT24 MT shared task test set, MINTADJUST ranks translations and systems more accurately than state-of-the-art metrics across a majority of language pairs, especially for high-quality systems. Furthermore, MINTADJUST outperforms AUTORANK, the ensembling method used by the organizers.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

34 of 57 references.

References · 34

  1. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  2. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  3. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  4. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  5. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  6. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  7. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  8. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 218
  9. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  10. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  11. 2022Metrics reloaded: Recommendations for image analysis validationcited by 195
  12. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
22 more
  1. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 89
  2. 2023Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibrationcited by 74
  3. 2019TIGEr: Text-to-Image Grounding for Image Caption Evaluationcited by 58
  4. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 58
  5. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  6. 2022Evaluation Gaps in Machine Learning Practicecited by 49
  7. 2024Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuraciescited by 40
  8. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  9. 2020Overinterpretation reveals image classification model pathologiescited by 28
  10. 2024Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracycited by 25
  11. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  12. 2023There's no Data Like Better Data: Using QE Metrics for MT Data Filteringcited by 19
  13. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  14. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  15. 2024Can Automatic Metrics Assess High-Quality Translations?cited by 16
  16. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 16
  17. 2021Reward Optimization for Neural Machine Translation with Learned Metricscited by 12
  18. 2024A Preference-driven Paradigm for Enhanced Translation with Large Language Modelscited by 9
  19. 2024MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibrationcited by 6
  20. 2022Adam Mickiewicz University at WMT 2022: NER-Assisted and Quality-Aware Neural Machine Translationcited by 5
  21. 2024Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysiscited by 4
  22. 2013A Contrario Selection of Optimal Partitions for Image Segmentationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Tower+: Bridging Generality and Translation Specialization in Multilingual LLMscited by 40
  2. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translationcited by 4
  3. 2025Investigating Test-Time Scaling with Reranking for Machine Translationcited by 3
  4. 2026Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domainscited by 2
  5. 2025Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotationscited by 1
  6. 2026CAT-Translate: Building Compact Open-Source Models for Japanese-English Translationcited by 0