mlsanityData to 7 Oct 2026

Evaluating Robustness to Input Perturbations for Neural Machine Translation

arXiv 2005.00580 ↗First posted 1 May 2020cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) models are sensitive to small perturbations in the input. Robustness to such perturbations is typically measured using translation quality metrics such as BLEU on the noisy input. This paper proposes additional metrics which measure the relative degradation and changes in translation when small perturbations are added to the input. We focus on a class of models employing subword regularization to address robustness and perform extensive evaluations of these models using the robustness measures proposed. Results show that our proposed metrics reveal a clear trend of improved robustness to perturbations when subword regularization methods are used.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 11; 2023: 7; 2024: 6; 2025: 1.

36 papers.

Sort:

By area:

  1. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  2. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 100
  3. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  4. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  5. 2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
  6. 2022MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translationcited by 38
  7. 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
  8. 2022Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrievalcited by 20
  9. 2021Interpreting the Robustness of Neural NLP Models to Textual Perturbationscited by 17
  10. 2021Efficient Inference for Multilingual Neural Machine Translationcited by 10
  11. 2021On the Robustness of Intent Classification and Slot Labeling in Goal-oriented Dialog Systems to Real-world Noisecited by 9
  12. 2024An Empirical Study of In-context Learning in LLMs for Machine Translationcited by 9
24 more
  1. 2022MaxMatch-Dropout: Subword Regularization for WordPiececited by 8
  2. 2023PGODE: Towards High-quality System Dynamics Modelingcited by 7
  3. 2022Hierarchical Interpretation of Neural Text Classificationcited by 5
  4. 2024Sentence-level Aggregation of Lexical Metrics Correlates Stronger with Human Judgements than Corpus-level Aggregationcited by 5
  5. 2022On the data requirements of probingcited by 4
  6. 2022Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translationcited by 4
  7. 2022Improving the Robustness of Summarization Models by Detecting and Removing Input Noisecited by 4
  8. 2023A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translationcited by 4
  9. 2021Sometimes We Want Translationesecited by 3
  10. 2022Rare but Severe Neural Machine Translation Errors Induced by Minimal Deletion: An Empirical Study on Chinese and Englishcited by 3
  11. 2022Checks and Strategies for Enabling Code-Switched Machine Translationcited by 3
  12. 2025Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generationcited by 3
  13. 2021Faithful Target Attribute Prediction in Neural Machine Translationcited by 2
  14. 2024Challenges and Opportunities in Text Generation Explainabilitycited by 2
  15. 2021Understanding Pre-Editing for Black-Box Neural Machine Translationcited by 1
  16. 2021Doubly-Trained Adversarial Data Augmentation for Neural Machine Translationcited by 1
  17. 2024Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognitioncited by 1
  18. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  19. 2021Masked Adversarial Generation for Neural Machine Translationcited by 0
  20. 2022Machine Translation Robustness to Natural Asemantic Variationcited by 0
  21. 2023Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?cited by 0
  22. 2023Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variationscited by 0
  23. 2023An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contributioncited by 0
  24. 2024NMT-Obfuscator Attack: Ignore a sentence in translation with only one wordcited by 0