mlsanityData to 14 Oct 2026

MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration

arXiv 2411.00390 ↗First posted 1 Nov 2024cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We present MetaMetrics-MT, an innovative metric designed to evaluate machine translation (MT) tasks by aligning closely with human preferences through Bayesian optimization with Gaussian Processes. MetaMetrics-MT enhances existing MT metrics by optimizing their correlation with human judgments. Our experiments on the WMT24 metric shared task dataset demonstrate that MetaMetrics-MT outperforms all existing baselines, setting a new benchmark for state-of-the-art performance in the reference-based setting. Furthermore, it achieves comparable results to leading metrics in the reference-free setting, offering greater efficiency.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

14 of 23 references.

References · 14

  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  3. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  4. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  5. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  6. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 146
  7. 2020Rethinking embedding coupling in pre-trained language modelscited by 103
  8. 2021Learning Compact Metrics for MTcited by 93
  9. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 87
  10. 2023Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibrationcited by 74
  11. 2024Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracycited by 25
  12. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
2 more
  1. 2024ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Modelscited by 20
  2. 2024MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferencescited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progresscited by 10
  2. 2025Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translationcited by 6
  3. 2025Reward Models are Metrics in a Trench Coatcited by 2
  4. 2025Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generationcited by 1
  5. 2025Simple and Effective Baselines for Code Summarisation Evaluationcited by 0
  6. 2026Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluationcited by 0