mlsanityData to 14 Oct 2026

Improving Neural Machine Translation Models with Monolingual Data

arXiv 1511.06709 ↗First posted 20 Nov 2015cs.CLcited by 788

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) has obtained state-of-the art performance for several language pairs, while only using parallel data for training. Target-side monolingual data plays an important role in boosting fluency for phrase-based statistical machine translation, and we investigate the use of monolingual data for NMT. In contrast to previous work, which combines NMT models with separately trained language models, we note that encoder-decoder NMT architectures already have the capacity to learn the same information as a language model, and we explore strategies to train with monolingual data without changing the neural network architecture. By pairing monolingual training data with an automatic back-translation, we can treat it as additional parallel training data, and we obtain substantial improvements on the WMT 15 task English<->German (+2.8-3.7 BLEU), and for the low-resourced IWSLT 14 task Turkish->English (+2.1-3.4 BLEU), obtaining new state-of-the-art results. We also show that fine-tuning on in-domain monolingual and parallel data gives substantial improvements for the IWSLT 15 task English->German.

Key facts. Cited by 788 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 209; 2022: 173; 2023: 146; 2024: 126; 2025: 74; 2026: 60.

788 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  2. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  3. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  4. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2021Editing Factual Knowledge in Language Modelscited by 553
  7. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  8. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  9. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  10. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  11. 2022Autoformalization with Large Language Modelscited by 263
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
38 more
  1. 2021A Survey on Deep Semi-supervised Learningcited by 251
  2. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  3. 2023Improving CLIP Training with Language Rewritescited by 217
  4. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  5. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  6. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  7. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  8. 2022A Review on Language Models as Knowledge Basescited by 157
  9. 2023Self-Alignment with Instruction Backtranslationcited by 151
  10. 2021Unsupervised Speech Recognitioncited by 148
  11. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  12. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  13. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  14. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 130
  15. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  16. 2021Scaling Laws for Neural Machine Translationcited by 111
  17. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  18. 2025Agentic Large Language Models, a surveycited by 108
  19. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  20. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  21. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  22. 2021Survey of Low-Resource Machine Translationcited by 92
  23. 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generationcited by 90
  24. 2022PEER: A Collaborative Language Modelcited by 86
  25. 2021Lessons on Parameter Sharing across Layers in Transformerscited by 85
  26. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  27. 2021Leveraging Automated Unit Tests for Unsupervised Code Translationcited by 82
  28. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  29. 2023VeCLIP: Improving CLIP Training via Visual-enriched Captionscited by 70
  30. 2024Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classificationcited by 70
  31. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  32. 2023Structure-informed Language Models Are Protein Designerscited by 68
  33. 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
  34. 2022Graph Data Augmentation for Graph Machine Learning: A Surveycited by 62
  35. 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
  36. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  37. 2022Democratizing Neural Machine Translation with OPUS-MTcited by 58
  38. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57

50 most cited of 788 All 788 (JSON)