# On the Language Coverage Bias for Neural Machine Translation

arXiv: https://arxiv.org/abs/2106.03297 · HTML: https://mlsanity.com/paper/2106.03297/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://mlsanity.com/paper/1901.10430/) (2019)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [BLEU might be Guilty but References are not Innocent](https://mlsanity.com/paper/2004.06063/) (2020)
- [Tagged Back-Translation](https://mlsanity.com/paper/1906.06442/) (2019)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1806.00258/) (2018)
- [The Effect of Translationese in Machine Translation Test Sets](https://mlsanity.com/paper/1906.08069/) (2019)
- [compare-mt: A Tool for Holistic Comparison of Language Generation Systems](https://mlsanity.com/paper/1903.07926/) (2019)
- [On The Evaluation of Machine Translation Systems Trained With Back-Translation](https://mlsanity.com/paper/1908.05204/) (2019)
- [Translationese as a Language in "Multilingual" NMT](https://mlsanity.com/paper/1911.03823/) (2019)
- [Post-editese: an Exacerbated Translationese](https://mlsanity.com/paper/1907.00900/) (2019)
- [Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation](https://mlsanity.com/paper/1911.03362/) (2019)
- [On Long-Tailed Phenomena in Neural Machine Translation](https://mlsanity.com/paper/2010.04924/) (2020)
- [Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation](https://mlsanity.com/paper/1808.09006/) (2018)
- [The Source-Target Domain Mismatch Problem in Machine Translation](https://mlsanity.com/paper/1909.13151/) (2019)

## Built on by (most cited first, 12 of 12)

- [Language Models are Good Translators](https://mlsanity.com/paper/2106.13627/) (2021)
- [On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation](https://mlsanity.com/paper/2110.01811/) (2021)
- [Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation](https://mlsanity.com/paper/2203.08442/) (2022)
- [Understanding Translationese in Cross-Lingual Summarization](https://mlsanity.com/paper/2212.07220/) (2022)
- [Understanding and Mitigating the Uncertainty in Zero-Shot Translation](https://mlsanity.com/paper/2205.10068/) (2022)
- [ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.03999/) (2022)
- [Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2203.08394/) (2022)
- [Translating away Translationese without Parallel Data](https://mlsanity.com/paper/2310.18830/) (2023)
- [Original or Translated? On the Use of Parallel Data for Translation Quality Estimation](https://mlsanity.com/paper/2212.10257/) (2022)
- [Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation](https://mlsanity.com/paper/2309.16599/) (2023)
- [Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering](https://mlsanity.com/paper/2406.02331/) (2024)
- [Pluggable Neural Machine Translation Models via Memory-augmented Adapters](https://mlsanity.com/paper/2307.06029/) (2023)

## Abstract (arXiv)

Language coverage bias, which indicates the content-dependent differences between sentence pairs originating from the source and target languages, is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the language coverage bias in the training data, and find that using only the source-original data achieves comparable performance with using full training data. Based on these observations, we further propose two simple and effective approaches to alleviate the language coverage bias problem through explicitly distinguishing between the source- and target-original training data, which consistently improve the performance over strong baselines on six WMT20 translation tasks. Complementary to the translationese effect, language coverage bias provides another explanation for the performance drop caused by back-translation. We also apply our approach to both back- and forward-translation and find that mitigating the language coverage bias can improve the performance of both the two representative data augmentation methods and their tagged variants.