# Evaluating Neural Machine Comprehension Model Robustness to Noisy Inputs and Adversarial Attacks

arXiv: https://arxiv.org/abs/2005.00190 · HTML: https://mlsanity.com/paper/2005.00190/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 3 of 3)

- [Enhancing Robustness of AI Offensive Code Generators via Data Augmentation](https://mlsanity.com/paper/2306.05079/) (2023)
- [Can NMT Understand Me? Towards Perturbation-based Evaluation of NMT Models for Code Generation](https://mlsanity.com/paper/2203.15319/) (2022)
- [Feedback Loops and Code Perturbations in LLM-based Software Engineering: A Case Study on a C-to-Rust Translation System](https://mlsanity.com/paper/2512.02567/) (2025)

## Abstract (arXiv)

We evaluate machine comprehension models' robustness to noise and adversarial attacks by performing novel perturbations at the character, word, and sentence level. We experiment with different amounts of perturbations to examine model confidence and misclassification rate, and contrast model performance in adversarial training with different embedding types on two benchmark datasets. We demonstrate improving model performance with ensembling. Finally, we analyze factors that effect model behavior under adversarial training and develop a model to predict model errors during adversarial attacks.