# RoBLEURT Submission for the WMT2021 Metrics Task

arXiv: https://arxiv.org/abs/2204.13352 · HTML: https://mlsanity.com/paper/2204.13352/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)

## Built on by (most cited first, 9 of 9)

- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics](https://mlsanity.com/paper/2212.05726/) (2022)
- [The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics](https://mlsanity.com/paper/2305.11806/) (2023)
- [MT-Ranker: Reference-free machine translation evaluation by inter-system ranking](https://mlsanity.com/paper/2401.17099/) (2024)
- [Alibaba-Translate China's Submission for WMT 2022 Metrics Shared Task](https://mlsanity.com/paper/2210.09683/) (2022)
- [Breeding Machine Translations: Evolutionary approach to survive and thrive in the world of automated evaluation](https://mlsanity.com/paper/2305.19330/) (2023)
- [Alibaba-Translate China's Submission for WMT 2022 Quality Estimation Shared Task](https://mlsanity.com/paper/2210.10049/) (2022)
- [HiMATE: A Hierarchical Multi-Agent Framework for Machine Translation Evaluation](https://mlsanity.com/paper/2505.16281/) (2025)
- [Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?](https://mlsanity.com/paper/2211.13865/) (2022)

## Abstract (arXiv)

In this paper, we present our submission to Shared Metrics Task: RoBLEURT (Robustly Optimizing the training of BLEURT). After investigating the recent advances of trainable metrics, we conclude several aspects of vital importance to obtain a well-performed metric model by: 1) jointly leveraging the advantages of source-included model and reference-only model, 2) continuously pre-training the model with massive synthetic data pairs, and 3) fine-tuning the model with data denoising strategy. Experimental results show that our model reaching state-of-the-art correlations with the WMT2020 human annotations upon 8 out of 10 to-English language pairs.