# Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach arXiv: https://arxiv.org/abs/2109.03645 ยท HTML: https://mlsanity.com/paper/2109.03645/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 51 references that are in the map. - First posted on arXiv 8 Sep 2021. ## Builds on (most cited first, 30 of 31) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018) - [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019) - [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017) - [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015) - [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016) - [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021) - [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017) - [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015) - [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021) - [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017) - [Data Augmentation for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1705.00440/) (2017) - [On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation](https://mlsanity.com/paper/2005.03642/) (2020) - [Analyzing Uncertainty in Neural Machine Translation](https://mlsanity.com/paper/1803.00047/) (2018) - [Reward Augmented Maximum Likelihood for Neural Structured Prediction](https://mlsanity.com/paper/1609.00150/) (2016) - [Data Noising as Smoothing in Neural Network Language Models](https://mlsanity.com/paper/1703.02573/) (2017) - [Tagged Back-Translation](https://mlsanity.com/paper/1906.06442/) (2019) - [Edinburgh Neural Machine Translation Systems for WMT 16](https://mlsanity.com/paper/1606.02891/) (2016) - [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018) - [SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation](https://mlsanity.com/paper/1808.07512/) (2018) - [A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation](https://mlsanity.com/paper/2009.13818/) (2020) - [Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation](https://mlsanity.com/paper/2010.10907/) (2020) - [Sequence-Level Mixed Sample Data Augmentation](https://mlsanity.com/paper/2011.09039/) (2020) - [Domain Robustness in Neural Machine Translation](https://mlsanity.com/paper/1911.03109/) (2019) - [Soft Contextual Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/1905.10523/) (2019) - [Trivial Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1809.00357/) (2018) - [Regularizing Neural Machine Translation by Target-bidirectional Agreement](https://mlsanity.com/paper/1808.04064/) (2018) - [Linguistic Input Features Improve Neural Machine Translation](https://mlsanity.com/paper/1606.02892/) (2016) - [Uncertainty-Aware Semantic Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2010.04411/) (2020) ## Built on by (most cited first, 8 of 8) - [Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences](https://mlsanity.com/paper/2305.09154/) (2023) - [TreeSwap: Data Augmentation for Machine Translation via Dependency Subtree Swapping](https://mlsanity.com/paper/2311.02355/) (2023) - [A Morphologically-Aware Dictionary-based Data Augmentation Technique for Machine Translation of Under-Represented Languages](https://mlsanity.com/paper/2402.01939/) (2024) - [Data Augmentation for Machine Translation via Dependency Subtree Swapping](https://mlsanity.com/paper/2307.07025/) (2023) - [Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation](https://mlsanity.com/paper/2401.16086/) (2024) - [Ukrainian-to-English folktale corpus: Parallel corpus creation and augmentation for machine translation in low-resource languages](https://mlsanity.com/paper/2410.10063/) (2024) - [Towards Cultural Bridge by Bahnaric-Vietnamese Translation Using Transfer Learning of Sequence-To-Sequence Pre-training Language Model](https://mlsanity.com/paper/2505.11421/) (2025) - [Leveraging Sentence-oriented Augmentation and Transformer-Based Architecture for Vietnamese-Bahnaric Translation](https://mlsanity.com/paper/2601.19124/) (2026) ## Abstract (arXiv) In the context of neural machine translation, data augmentation (DA) techniques may be used for generating additional training samples when the available parallel data are scarce. Many DA approaches aim at expanding the support of the empirical data distribution by generating new sentence pairs that contain infrequent words, thus making it closer to the true data distribution of parallel sentences. In this paper, we propose to follow a completely different approach and present a multi-task DA approach in which we generate new sentence pairs with transformations, such as reversing the order of the target sentence, which produce unfluent target sentences. During training, these augmented sentences are used as auxiliary tasks in a multi-task framework with the aim of providing new contexts where the target prefix is not informative enough to predict the next word. This strengthens the encoder and forces the decoder to pay more attention to the source representations of the encoder. Experiments carried out on six low-resource translation tasks show consistent improvements over the baseline and over DA methods aiming at extending the support of the empirical data distribution. The systems trained with our approach rely more on the source tokens, are more robust against domain shift and suffer less hallucinations.