# Human-Paraphrased References Improve Neural Machine Translation

arXiv: https://arxiv.org/abs/2010.10245 · HTML: https://mlsanity.com/paper/2010.10245/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [Reward Gaming in Conditional Text Generation](https://mlsanity.com/paper/2211.08714/) (2022)
- [Iterative Translation Refinement with Large Language Models](https://mlsanity.com/paper/2306.03856/) (2023)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References](https://mlsanity.com/paper/2305.15067/) (2023)
- [Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models](https://mlsanity.com/paper/2204.00471/) (2022)
- [Assessing Reference-Free Peer Evaluation for Machine Translation](https://mlsanity.com/paper/2104.05146/) (2021)
- [LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training](https://mlsanity.com/paper/2401.04348/) (2024)
- [Language Model Augmented Relevance Score](https://mlsanity.com/paper/2108.08485/) (2021)
- [PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models](https://mlsanity.com/paper/2409.12060/) (2024)
- [Variance-Aware Machine Translation Test Sets](https://mlsanity.com/paper/2111.04079/) (2021)
- [Rephrasing the Reference for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2211.16863/) (2022)
- [Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation](https://mlsanity.com/paper/2608.25089/) (2026)
- [On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation](https://mlsanity.com/paper/2110.03067/) (2021)
- [Form and Meaning in Intrinsic Multilingual Evaluations](https://mlsanity.com/paper/2601.10580/) (2026)
- [Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks](https://mlsanity.com/paper/2609.03734/) (2026)
- [Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation](https://mlsanity.com/paper/2610.19528/) (2026)

## Abstract (arXiv)

Automatic evaluation comparing candidate translations to human-generated paraphrases of reference translations has recently been proposed by Freitag et al. When used in place of original references, the paraphrased versions produce metric scores that correlate better with human judgment. This effect holds for a variety of different automatic metrics, and tends to favor natural formulations over more literal (translationese) ones. In this paper we compare the results of performing end-to-end system development using standard and paraphrased references. With state-of-the-art English-German NMT components, we show that tuning to paraphrased references produces a system that is significantly better according to human judgment, but 5 BLEU points worse when tested on standard references. Our work confirms the finding that paraphrased references yield metric scores that correlate better with human judgment, and demonstrates for the first time that using these scores for system development can lead to significant improvements.