# AdvAug: Robust Adversarial Augmentation for Neural Machine Translation

arXiv: https://arxiv.org/abs/2006.11834 · HTML: https://mlsanity.com/paper/2006.11834/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2020.

## Built on by (most cited first, 30 of 67)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Regularizing Generative Adversarial Networks under Limited Data](https://mlsanity.com/paper/2104.03310/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [Pyramid Adversarial Training Improves ViT Performance](https://mlsanity.com/paper/2111.15121/) (2021)
- [Faster Meta Update Strategy for Noise-Robust Deep Learning](https://mlsanity.com/paper/2104.15092/) (2021)
- [Evaluating Saliency Methods for Neural Language Models](https://mlsanity.com/paper/2104.05824/) (2021)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends](https://mlsanity.com/paper/2409.14457/) (2024)
- [A Survey of Adversarial Defences and Robustness in NLP](https://mlsanity.com/paper/2203.06414/) (2022)
- [Reweighting Augmented Samples by Minimizing the Maximal Expected Loss](https://mlsanity.com/paper/2103.08933/) (2021)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives](https://mlsanity.com/paper/2206.02457/) (2022)
- [TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding](https://mlsanity.com/paper/2205.06153/) (2022)
- [White-Box Multi-Objective Adversarial Attack on Dialogue Generation](https://mlsanity.com/paper/2305.03655/) (2023)
- [Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2204.06812/) (2022)
- [Self-supervised and Supervised Joint Training for Resource-rich Machine Translation](https://mlsanity.com/paper/2106.04060/) (2021)
- [TransFool: An Adversarial Attack against Neural Machine Translation Models](https://mlsanity.com/paper/2302.00944/) (2023)
- [Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax](https://mlsanity.com/paper/2105.13608/) (2021)
- [DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification](https://mlsanity.com/paper/2209.05297/) (2022)
- [Targeted Adversarial Attacks against Neural Machine Translation](https://mlsanity.com/paper/2303.01068/) (2023)
- [Improving the Robustness of Summarization Systems with Dual Augmentation](https://mlsanity.com/paper/2306.01090/) (2023)
- [As Easy as 1, 2, 3: Behavioural Testing of NMT Systems for Numerical Translation](https://mlsanity.com/paper/2107.08357/) (2021)
- [BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation](https://mlsanity.com/paper/2204.07837/) (2022)
- [Interdisciplinary Fairness in Imbalanced Research Proposal Topic Inference: A Hierarchical Transformer-based Method with Selective Interpolation](https://mlsanity.com/paper/2309.01717/) (2023)
- [Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation](https://mlsanity.com/paper/2203.07627/) (2022)
- [miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings](https://mlsanity.com/paper/2211.04928/) (2022)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [DSRM: Boost Textual Adversarial Training with Distribution Shift Risk Minimization](https://mlsanity.com/paper/2306.15164/) (2023)
- [Addressing the Vulnerability of NMT in Input Perturbations](https://mlsanity.com/paper/2104.09810/) (2021)

## Abstract (arXiv)

In this paper, we propose a new adversarial augmentation method for Neural Machine Translation (NMT). The main idea is to minimize the vicinal risk over virtual sentences sampled from two vicinity distributions, of which the crucial one is a novel vicinity distribution for adversarial sentences that describes a smooth interpolated embedding space centered around observed training sentence pairs. We then discuss our approach, AdvAug, to train NMT models using the embeddings of virtual sentences in sequence-to-sequence learning. Experiments on Chinese-English, English-French, and English-German translation benchmarks show that AdvAug achieves significant improvements over the Transformer (up to 4.9 BLEU points), and substantially outperforms other data augmentation techniques (e.g. back-translation) without using extra corpora.