mlsanityData to 7 Oct 2026

Addressing the Vulnerability of NMT in Input Perturbations

arXiv 2104.09810 ↗First posted 20 Apr 2021cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) has achieved significant breakthrough in performance but is known to suffer vulnerability to input perturbations. As real input noise is difficult to predict during training, robustness is a big issue for system deployment. In this paper, we improve the robustness of NMT models by reducing the effect of noisy words through a Context-Enhanced Reconstruction (CER) approach. CER trains the model to resist noise in two steps: (1) perturbation step that breaks the naturalness of input sequence with made-up words; (2) reconstruction step that defends the noise propagation by generating better and more robust contextual representation. Experimental results on Chinese-English (ZH-EN) and French-English (FR-EN) translation tasks demonstrate robustness improvement on both news and social media text. Further fine-tuning experiments on social media text show our approach can converge at a higher position and provide a better adaptation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 20 Apr 2021.

Builds on

29 of 42 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  9. 2018Generating Natural Language Adversarial Examplescited by 422
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  11. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  12. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
17 more
  1. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  2. 2019Robust Neural Machine Translation with Doubly Adversarial Inputscited by 105
  3. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  4. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  5. 2020AdvAug: Robust Adversarial Augmentation for Neural Machine Translationcited by 66
  6. 2020It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbationscited by 64
  7. 2018MTNT: A Testbed for Machine Translation of Noisy Textcited by 54
  8. 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
  9. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  10. 2019Improving Robustness of Machine Translation with Synthetic Noisecited by 36
  11. 2018Towards Robust Neural Machine Translationcited by 35
  12. 2020PiRhDy: Learning Pitch-, Rhythm-, and Dynamics-aware Embeddings for Symbolic Musiccited by 24
  13. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 22
  14. 2020Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encodingcited by 22
  15. 2018Robust Neural Machine Translation with Joint Textual and Phonetic Embeddingcited by 19
  16. 2019Findings of the First Shared Task on Machine Translation Robustnesscited by 17
  17. 2019CUNI System for the WMT19 Robustness Taskcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 1.

6 papers.

  1. 2024An Empirical Study of In-context Learning in LLMs for Machine Translationcited by 9
  2. 2023Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalizationcited by 3
  3. 2022Categorizing Semantic Representations for Neural Machine Translationcited by 2
  4. 2023Layer-wise Representation Fusion for Compositional Generalizationcited by 2
  5. 2025A Representation Level Analysis of NMT Model Robustness to Grammatical Errorscited by 1
  6. 2022Improving Lexical Embeddings for Robust Question Answeringcited by 0