# Residue-Based Natural Language Adversarial Attack Detection

arXiv: https://arxiv.org/abs/2204.10192 · HTML: https://mlsanity.com/paper/2204.10192/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 66 references that are in the map.
- First posted on arXiv 17 Apr 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Detecting Adversarial Samples from Artifacts](https://mlsanity.com/paper/1703.00410/) (2017)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Understanding Measures of Uncertainty for Adversarial Example Detection](https://mlsanity.com/paper/1803.08533/) (2018)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples](https://mlsanity.com/paper/1803.01128/) (2018)
- [Towards Improving Adversarial Training of NLP Models](https://mlsanity.com/paper/2109.00544/) (2021)
- [Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics](https://mlsanity.com/paper/1612.07767/) (2016)
- [Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors](https://mlsanity.com/paper/1909.06872/) (2019)
- [Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification](https://mlsanity.com/paper/1909.03084/) (2019)
- [Towards Crafting Text Adversarial Samples](https://mlsanity.com/paper/1707.02812/) (2017)
- [Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples](https://mlsanity.com/paper/2004.05887/) (2020)
- [Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge](https://mlsanity.com/paper/1808.08609/) (2018)
- [Robust Machine Comprehension Models via Adversarial Training](https://mlsanity.com/paper/1804.06473/) (2018)

## Built on by (most cited first, 8 of 8)

- [Identifying Adversarially Attackable and Robust Samples](https://mlsanity.com/paper/2301.12896/) (2023)
- [Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness](https://mlsanity.com/paper/2311.09694/) (2023)
- [Self-Supervised Representation Learning for Adversarial Attack Detection](https://mlsanity.com/paper/2407.04382/) (2024)
- [Sample Attackability in Natural Language Adversarial Attacks](https://mlsanity.com/paper/2306.12043/) (2023)
- [NLP Security and Ethics, in the Wild](https://mlsanity.com/paper/2504.06669/) (2025)
- [Sentiment Perception Adversarial Attacks on Neural Machine Translation Systems](https://mlsanity.com/paper/2305.01437/) (2023)
- [Gender Bias and Universal Substitution Adversarial Attacks on Grammatical Error Correction Systems for Automated Assessment](https://mlsanity.com/paper/2208.09466/) (2022)
- [VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations](https://mlsanity.com/paper/2306.01273/) (2023)

## Abstract (arXiv)

Deep learning based systems are susceptible to adversarial attacks, where a small, imperceptible change at the input alters the model prediction. However, to date the majority of the approaches to detect these attacks have been designed for image processing systems. Many popular image adversarial detection approaches are able to identify adversarial examples from embedding feature spaces, whilst in the NLP domain existing state of the art detection approaches solely focus on input text features, without consideration of model embedding spaces. This work examines what differences result when porting these image designed strategies to Natural Language Processing (NLP) tasks - these detectors are found to not port over well. This is expected as NLP systems have a very different form of input: discrete and sequential in nature, rather than the continuous and fixed size inputs for images. As an equivalent model-focused NLP detection approach, this work proposes a simple sentence-embedding "residue" based detector to identify adversarial examples. On many tasks, it out-performs ported image domain detectors and recent state of the art NLP specific detectors.