# Undersensitivity in Neural Reading Comprehension

arXiv: https://arxiv.org/abs/2003.04808 · HTML: https://mlsanity.com/paper/2003.04808/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2020.

## Built on by (most cited first, 9 of 9)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models](https://mlsanity.com/paper/2109.02837/) (2021)
- [Lexical Generalization Improves with Larger Models and Longer Training](https://mlsanity.com/paper/2210.12673/) (2022)
- [Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification](https://mlsanity.com/paper/2109.04385/) (2021)
- [Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models](https://mlsanity.com/paper/2210.11498/) (2022)
- [Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models](https://mlsanity.com/paper/2307.12507/) (2023)
- [Model Agnostic Answer Reranking System for Adversarial Question Answering](https://mlsanity.com/paper/2102.03016/) (2021)
- [Pay Attention to Real World Perturbations! Natural Robustness Evaluation in Machine Reading Comprehension](https://mlsanity.com/paper/2502.16523/) (2025)
- [An Understanding-Oriented Robust Machine Reading Comprehension Model](https://mlsanity.com/paper/2207.00187/) (2022)

## Abstract (arXiv)

Current reading comprehension models generalise well to in-distribution test sets, yet perform poorly on adversarially selected inputs. Most prior work on adversarial inputs studies oversensitivity: semantically invariant text perturbations that cause a model's prediction to change when it should not. In this work we focus on the complementary problem: excessive prediction undersensitivity, where input text is meaningfully changed but the model's prediction does not, even though it should. We formulate a noisy adversarial attack which searches among semantic variations of the question for which a model erroneously predicts the same answer, and with even higher probability. Despite comprising unanswerable questions, both SQuAD2.0 and NewsQA models are vulnerable to this attack. This indicates that although accurate, models tend to rely on spurious patterns and do not fully consider the information specified in a question. We experiment with data augmentation and adversarial training as defences, and find that both substantially decrease vulnerability to attacks on held out data, as well as held out attack spaces. Addressing undersensitivity also improves results on AddSent and AddOneSent, and models furthermore generalise better when facing train/evaluation distribution mismatch: they are less prone to overly rely on predictive cues present only in the training set, and outperform a conventional model by as much as 10.9% F1.