# Decoding and Diversity in Machine Translation

arXiv: https://arxiv.org/abs/2011.13477 · HTML: https://mlsanity.com/paper/2011.13477/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Detection and Measurement of Syntactic Templates in Generated Text](https://mlsanity.com/paper/2407.00211/) (2024)
- [First the worst: Finding better gender translations during beam search](https://mlsanity.com/paper/2104.07429/) (2021)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training](https://mlsanity.com/paper/2305.02031/) (2023)
- [ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation](https://mlsanity.com/paper/2308.02223/) (2023)
- [How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation](https://mlsanity.com/paper/2105.13782/) (2021)
- [To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation](https://mlsanity.com/paper/2401.01419/) (2024)
- [RetroDiff: Retrosynthesis as Multi-stage Distribution Interpolation](https://mlsanity.com/paper/2311.14077/) (2023)
- [Learning Evaluation Models from Large Language Models for Sequence Generation](https://mlsanity.com/paper/2308.04386/) (2023)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [On Synthetic Data for Back Translation](https://mlsanity.com/paper/2310.13675/) (2023)
- [Anti-LM Decoding for Zero-shot In-context Machine Translation](https://mlsanity.com/paper/2311.08324/) (2023)
- [Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model](https://mlsanity.com/paper/2511.02958/) (2025)

## Abstract (arXiv)

Neural Machine Translation (NMT) systems are typically evaluated using automated metrics that assess the agreement between generated translations and ground truth candidates. To improve systems with respect to these metrics, NLP researchers employ a variety of heuristic techniques, including searching for the conditional mode (vs. sampling) and incorporating various training heuristics (e.g., label smoothing). While search strategies significantly improve BLEU score, they yield deterministic outputs that lack the diversity of human translations. Moreover, search tends to bias the distribution of translated gender pronouns. This makes human-level BLEU a misleading benchmark in that modern MT systems cannot approach human-level BLEU while simultaneously maintaining human-level translation diversity. In this paper, we characterize distributional differences between generated and real translations, examining the cost in diversity paid for the BLEU scores enjoyed by NMT. Moreover, our study implicates search as a salient source of known bias when translating gender pronouns.