# Generalizing Back-Translation in Neural Machine Translation

arXiv: https://arxiv.org/abs/1906.07286 · HTML: https://mlsanity.com/paper/1906.07286/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2019.

## Built on by (most cited first, 18 of 18)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [Back-translation for Large-Scale Multilingual Machine Translation](https://mlsanity.com/paper/2109.08712/) (2021)
- [Comparison of Grammatical Error Correction Using Back-Translation Models](https://mlsanity.com/paper/2104.07848/) (2021)
- [Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation](https://mlsanity.com/paper/2502.20420/) (2025)
- [Is Encoder-Decoder Redundant for Neural Machine Translation?](https://mlsanity.com/paper/2210.11807/) (2022)
- [Rethinking Label Smoothing on Multi-hop Question Answering](https://mlsanity.com/paper/2212.09512/) (2022)
- [On Synthetic Data for Back Translation](https://mlsanity.com/paper/2310.13675/) (2023)
- [Improving Language Model Integration for Neural Machine Translation](https://mlsanity.com/paper/2306.05077/) (2023)
- [CantonMT: Cantonese to English NMT Platform with Fine-Tuned Models Using Synthetic Back-Translation Data](https://mlsanity.com/paper/2403.11346/) (2024)
- [End-to-End Training for Back-Translation with Categorical Reparameterization Trick](https://mlsanity.com/paper/2202.08465/) (2022)
- [Enhancing Personality Recognition in Dialogue by Data Augmentation and Heterogeneous Conversational Graph Networks](https://mlsanity.com/paper/2401.05871/) (2024)
- [CANTONMT: Investigating Back-Translation and Model-Switch Mechanisms for Cantonese-English Neural Machine Translation](https://mlsanity.com/paper/2405.08172/) (2024)
- [Exploring the traditional NMT model and Large Language Model for chat translation](https://mlsanity.com/paper/2409.16331/) (2024)
- [Multi-Hypothesis Distillation of Multilingual Neural Translation Models for Low-Resource Languages](https://mlsanity.com/paper/2507.21568/) (2025)
- [Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction](https://mlsanity.com/paper/2604.07320/) (2026)
- [Extracting Problem and Method Sentence from Scientific Papers: A Context-enhanced Transformer Using Formulaic Expression Desensitization](https://mlsanity.com/paper/2606.26481/) (2026)

## Abstract (arXiv)

Back-translation - data augmentation by translating target monolingual data - is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German - English news translation task.