# Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation

arXiv: https://arxiv.org/abs/1808.09006 · HTML: https://mlsanity.com/paper/1808.09006/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2018.

## Built on by (most cited first, 19 of 19)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation](https://mlsanity.com/paper/2106.00903/) (2021)
- [Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation](https://mlsanity.com/paper/2106.00941/) (2021)
- [On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation](https://mlsanity.com/paper/2110.01811/) (2021)
- [Improving Simultaneous Machine Translation with Monolingual Data](https://mlsanity.com/paper/2212.01188/) (2022)
- [On the Language Coverage Bias for Neural Machine Translation](https://mlsanity.com/paper/2106.03297/) (2021)
- [Domain-Specific Text Generation for Machine Translation](https://mlsanity.com/paper/2208.05909/) (2022)
- [Exploiting Curriculum Learning in Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2109.11177/) (2021)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation](https://mlsanity.com/paper/2210.07054/) (2022)
- [On Synthetic Data for Back Translation](https://mlsanity.com/paper/2310.13675/) (2023)
- [RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts](https://mlsanity.com/paper/2406.06577/) (2024)
- [Can Synthetic Translations Improve Bitext Quality?](https://mlsanity.com/paper/2203.07643/) (2022)
- [Robust Guidance for Unsupervised Data Selection: Capturing Perplexing Named Entities for Domain-Specific Machine Translation](https://mlsanity.com/paper/2402.19267/) (2024)
- [From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2412.00966/) (2024)
- [Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling](https://mlsanity.com/paper/2510.08245/) (2025)
- [End-to-End Training for Back-Translation with Categorical Reparameterization Trick](https://mlsanity.com/paper/2202.08465/) (2022)
- [High-Quality Data Augmentation for Low-Resource NMT: Combining a Translation Memory, a GAN Generator, and Filtering](https://mlsanity.com/paper/2408.12079/) (2024)
- [From Scarcity to Efficiency: Investigating the Effects of Data Augmentation on African Machine Translation](https://mlsanity.com/paper/2509.07471/) (2025)

## Abstract (arXiv)

Neural Machine Translation has achieved state-of-the-art performance for several language pairs using a combination of parallel and synthetic data. Synthetic data is often generated by back-translating sentences randomly sampled from monolingual data using a reverse translation model. While back-translation has been shown to be very effective in many cases, it is not entirely clear why. In this work, we explore different aspects of back-translation, and show that words with high prediction loss during training benefit most from the addition of synthetic data. We introduce several variations of sampling strategies targeting difficult-to-predict words using prediction losses and frequencies of words. In addition, we also target the contexts of difficult words and sample sentences that are similar in context. Experimental results for the WMT news translation task show that our method improves translation quality by up to 1.7 and 1.2 Bleu points over back-translation using random sampling for German-English and English-German, respectively.