# Using Monolingual Data in Neural Machine Translation: a Systematic Study

arXiv: https://arxiv.org/abs/1903.11437 · HTML: https://mlsanity.com/paper/1903.11437/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 21 of 21)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT](https://mlsanity.com/paper/2109.01396/) (2021)
- [Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu](https://mlsanity.com/paper/2502.11862/) (2025)
- [Domain-Specific Text Generation for Machine Translation](https://mlsanity.com/paper/2208.05909/) (2022)
- [Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance](https://mlsanity.com/paper/2205.02293/) (2022)
- [Lost in Literalism: How Supervised Training Shapes Translationese in LLMs](https://mlsanity.com/paper/2503.04369/) (2025)
- [NewTerm: Benchmarking Real-Time New Terms for Large Language Models with Annual Updates](https://mlsanity.com/paper/2410.20814/) (2024)
- [Code Generation from Natural Language with Less Prior and More Monolingual Data](https://mlsanity.com/paper/2101.00259/) (2021)
- [Crowdsourced Phrase-Based Tokenization for Low-Resourced Neural Machine Translation: The Case of Fon Language](https://mlsanity.com/paper/2103.08052/) (2021)
- [PuoBERTa: Training and evaluation of a curated language model for Setswana](https://mlsanity.com/paper/2310.09141/) (2023)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [Rule-Based, Neural and LLM Back-Translation: Comparative Insights from a Variant of Ladin](https://mlsanity.com/paper/2407.08819/) (2024)
- [Exploring Unsupervised Pretraining Objectives for Machine Translation](https://mlsanity.com/paper/2106.05634/) (2021)
- [HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints](https://mlsanity.com/paper/2109.04443/) (2021)
- [How Low is Too Low? A Computational Perspective on Extremely Low-Resource Languages](https://mlsanity.com/paper/2105.14515/) (2021)
- [Translation Asymmetry in LLMs as a Data Augmentation Factor: A Case Study for 6 Romansh Language Varieties](https://mlsanity.com/paper/2603.25489/) (2026)
- [Leveraging Synthetic Targets for Machine Translation](https://mlsanity.com/paper/2305.06155/) (2023)
- [TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation](https://mlsanity.com/paper/2508.08680/) (2025)
- [Empirical Analysis of Korean Public AI Hub Parallel Corpora and in-depth Analysis using LIWC](https://mlsanity.com/paper/2110.15023/) (2021)
- [Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data](https://mlsanity.com/paper/2504.21747/) (2025)
- [Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction](https://mlsanity.com/paper/2604.07320/) (2026)

## Abstract (arXiv)

Neural Machine Translation (MT) has radically changed the way systems are developed. A major difference with the previous generation (Phrase-Based MT) is the way monolingual target data, which often abounds, is used in these two paradigms. While Phrase-Based MT can seamlessly integrate very large language models trained on billions of sentences, the best option for Neural MT developers seems to be the generation of artificial parallel data through textslback-translation - a technique that fails to fully take advantage of existing datasets. In this paper, we conduct a systematic study of back-translation, comparing alternative uses of monolingual data, as well as multiple data generation procedures. Our findings confirm that back-translation is very effective and give new explanations as to why this is the case. We also introduce new data simulation techniques that are almost as effective, yet much cheaper to implement.