# "That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks

arXiv: https://arxiv.org/abs/2204.04636 · HTML: https://mlsanity.com/paper/2204.04636/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 10 Apr 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Adversarial Logit Pairing](https://mlsanity.com/paper/1803.06373/) (2018)
- [Certified Robustness to Adversarial Word Substitutions](https://mlsanity.com/paper/1909.00986/) (2019)
- [Combating Adversarial Misspellings with Robust Word Recognition](https://mlsanity.com/paper/1905.11268/) (2019)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [The Odds are Odd: A Statistical Test for Detecting Adversarial Examples](https://mlsanity.com/paper/1902.04818/) (2019)
- [Towards Robustness Against Natural Language Word Substitutions](https://mlsanity.com/paper/2107.13541/) (2021)
- [Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification](https://mlsanity.com/paper/1909.03084/) (2019)
- [Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples](https://mlsanity.com/paper/2004.05887/) (2020)
- [When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures](https://mlsanity.com/paper/1909.03418/) (2019)
- [Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples](https://mlsanity.com/paper/1810.11580/) (2018)
- [Early Methods for Detecting Adversarial Images](https://mlsanity.com/paper/1608.00530/) (2016)
- [Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble](https://mlsanity.com/paper/2006.11627/) (2020)
- [Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection](https://mlsanity.com/paper/1905.09186/) (2019)
- [Detection Defense Against Adversarial Attacks with Saliency Map](https://mlsanity.com/paper/2009.02738/) (2020)

## Built on by (most cited first, 24 of 24)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents](https://mlsanity.com/paper/2411.09523/) (2024)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [Why do universal adversarial attacks work on large language models?: Geometry might be the answer](https://mlsanity.com/paper/2309.00254/) (2023)
- [Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions](https://mlsanity.com/paper/2503.14043/) (2025)
- [TextShield: Beyond Successfully Detecting Adversarial Sentences in Text Classification](https://mlsanity.com/paper/2302.02023/) (2023)
- [IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models](https://mlsanity.com/paper/2303.03124/) (2023)
- [Masked Language Model Based Textual Adversarial Example Detection](https://mlsanity.com/paper/2304.08767/) (2023)
- [ROIC-DM: Robust Text Inference and Classification via Diffusion Model](https://mlsanity.com/paper/2401.03514/) (2024)
- [Exploring Adversarial Attacks and Defenses in Vision Transformers trained with DINO](https://mlsanity.com/paper/2206.06761/) (2022)
- [An Interpretability Evaluation Benchmark for Pre-trained Language Models](https://mlsanity.com/paper/2207.13948/) (2022)
- [Exploring Gradient-Guided Masked Language Model to Detect Textual Adversarial Attacks](https://mlsanity.com/paper/2504.08798/) (2025)
- [SemRoDe: Macro Adversarial Training to Learn Representations That are Robust to Word-Level Attacks](https://mlsanity.com/paper/2403.18423/) (2024)
- [ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation](https://mlsanity.com/paper/2210.12396/) (2022)
- [On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection](https://mlsanity.com/paper/2306.15705/) (2023)
- [Guided Perturbation Sensitivity (GPS): Detecting Adversarial Text via Embedding Stability and Word Importance](https://mlsanity.com/paper/2508.11667/) (2025)
- [Alert-ME: An Explainability-Driven Defense Against Adversarial Examples in Transformer-Based Text Classification](https://mlsanity.com/paper/2307.01225/) (2023)
- [DA3: A Distribution-Aware Adversarial Attack against Language Models](https://mlsanity.com/paper/2311.08598/) (2023)
- [Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation](https://mlsanity.com/paper/2504.13551/) (2025)
- [VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations](https://mlsanity.com/paper/2306.01273/) (2023)
- [RedHerring Attack: Testing the Reliability of Attack Detection](https://mlsanity.com/paper/2509.20691/) (2025)
- [FeatureLens: A Highly Generalizable and Interpretable Framework for Detecting Adversarial Examples Based on Image Features](https://mlsanity.com/paper/2512.03625/) (2025)
- [Rectifying Adversarial Examples Using Their Vulnerabilities](https://mlsanity.com/paper/2601.00270/) (2026)
- [RTD-Guard: A Black-Box Textual Adversarial Detection Framework via Replacement Token Detection](https://mlsanity.com/paper/2603.12582/) (2026)

## Abstract (arXiv)

Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adversarial text examples. The approach identifies patterns in the logits of the target classifier when perturbing the input text. The proposed detector improves the current state-of-the-art performance in recognizing adversarial inputs and exhibits strong generalization capabilities across different NLP models, datasets, and word-level attacks.