# Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models

arXiv: https://arxiv.org/abs/1809.02079 · HTML: https://mlsanity.com/paper/1809.02079/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 26 of 26)

- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://mlsanity.com/paper/2302.13007/) (2023)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching](https://mlsanity.com/paper/2303.16756/) (2023)
- [Robust Conversational Agents against Imperceptible Toxicity Triggers](https://mlsanity.com/paper/2205.02392/) (2022)
- [Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery](https://mlsanity.com/paper/2303.01962/) (2023)
- [A Differentiable Language Model Adversarial Attack on Text Classifiers](https://mlsanity.com/paper/2107.11275/) (2021)
- [Residue-Based Natural Language Adversarial Attack Detection](https://mlsanity.com/paper/2204.10192/) (2022)
- [EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification](https://mlsanity.com/paper/2204.11205/) (2022)
- [Robustness and Sensitivity of BERT Models Predicting Alzheimer's Disease from Text](https://mlsanity.com/paper/2109.11888/) (2021)
- [Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study](https://mlsanity.com/paper/2210.10678/) (2022)
- [Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models](https://mlsanity.com/paper/2210.11498/) (2022)
- [xSIM++: An Improved Proxy to Bitext Mining Performance for Low-Resource Languages](https://mlsanity.com/paper/2306.12907/) (2023)
- [Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models](https://mlsanity.com/paper/2307.12507/) (2023)
- [Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models](https://mlsanity.com/paper/2404.03577/) (2024)
- [Tribrid: Stance Classification with Neural Inconsistency Detection](https://mlsanity.com/paper/2109.06508/) (2021)
- [Enhancing LLM Knowledge Learning through Generalization](https://mlsanity.com/paper/2503.03705/) (2025)
- [BootAug: Boosting Text Augmentation via Hybrid Instance Filtering Framework](https://mlsanity.com/paper/2210.02941/) (2022)
- [Adversarial Robustness of Visual Dialog](https://mlsanity.com/paper/2207.02639/) (2022)
- [Catch Me If You Can: Deceiving Stance Detection and Geotagging Models to Protect Privacy of Individuals on Twitter](https://mlsanity.com/paper/2207.11500/) (2022)
- [A Machine Learning Approach to Detect Customer Satisfaction From Multiple Tweet Parameters](https://mlsanity.com/paper/2402.15992/) (2024)
- [White-Box Sensitivity Auditing with Steering Vectors](https://mlsanity.com/paper/2601.16398/) (2026)
- [Structure-Preserving Document-Level Data Augmentation via Discourse Parsing](https://mlsanity.com/paper/2610.19497/) (2026)

## Abstract (arXiv)

We present two categories of model-agnostic adversarial strategies that reveal the weaknesses of several generative, task-oriented dialogue models: Should-Not-Change strategies that evaluate over-sensitivity to small and semantics-preserving edits, as well as Should-Change strategies that test if a model is over-stable against subtle yet semantics-changing modifications. We next perform adversarial training with each strategy, employing a max-margin approach for negative generative examples. This not only makes the target dialogue model more robust to the adversarial inputs, but also helps it perform significantly better on the original inputs. Moreover, training on all strategies combined achieves further improvements, achieving a new state-of-the-art performance on the original task (also verified via human evaluation). In addition to adversarial training, we also address the robustness task at the model-level, by feeding it subword units as both inputs and outputs, and show that the resulting model is equally competitive, requires only 1/4 of the original vocabulary size, and is robust to one of the adversarial strategies (to which the original model is vulnerable) even without adversarial training.