# MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration

arXiv: https://arxiv.org/abs/2411.00390 · HTML: https://mlsanity.com/paper/2411.00390/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [Rethinking embedding coupling in pre-trained language models](https://mlsanity.com/paper/2010.12821/) (2020)
- [Learning Compact Metrics for MT](https://mlsanity.com/paper/2110.06341/) (2021)
- [Larger-Scale Transformers for Multilingual Masked Language Modeling](https://mlsanity.com/paper/2105.00572/) (2021)
- [Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration](https://mlsanity.com/paper/2305.14324/) (2023)
- [Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy](https://mlsanity.com/paper/2409.09598/) (2024)
- [Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey](https://mlsanity.com/paper/2409.11564/) (2024)
- [ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models](https://mlsanity.com/paper/2406.09334/) (2024)
- [MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences](https://mlsanity.com/paper/2410.02381/) (2024)

## Built on by (most cited first, 6 of 6)

- [Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress](https://mlsanity.com/paper/2506.19571/) (2025)
- [Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translation](https://mlsanity.com/paper/2503.08327/) (2025)
- [Reward Models are Metrics in a Trench Coat](https://mlsanity.com/paper/2510.03231/) (2025)
- [Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generation](https://mlsanity.com/paper/2508.18709/) (2025)
- [Simple and Effective Baselines for Code Summarisation Evaluation](https://mlsanity.com/paper/2505.19392/) (2025)
- [Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation](https://mlsanity.com/paper/2605.09098/) (2026)

## Abstract (arXiv)

We present MetaMetrics-MT, an innovative metric designed to evaluate machine translation (MT) tasks by aligning closely with human preferences through Bayesian optimization with Gaussian Processes. MetaMetrics-MT enhances existing MT metrics by optimizing their correlation with human judgments. Our experiments on the WMT24 metric shared task dataset demonstrate that MetaMetrics-MT outperforms all existing baselines, setting a new benchmark for state-of-the-art performance in the reference-based setting. Furthermore, it achieves comparable results to leading metrics in the reference-free setting, offering greater efficiency.