# Deep Text Classification Can be Fooled

arXiv: https://arxiv.org/abs/1704.08006 · HTML: https://mlsanity.com/paper/1704.08006/

## Key facts

- Cited by 106 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 30 of 106)

- [Poisoning Retrieval Corpora by Injecting Adversarial Passages](https://mlsanity.com/paper/2310.19156/) (2023)
- [BERT: A Review of Applications in Natural Language Processing and Understanding](https://mlsanity.com/paper/2103.11943/) (2021)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [Towards Robustness Against Natural Language Word Substitutions](https://mlsanity.com/paper/2107.13541/) (2021)
- [Certified Robustness to Text Adversarial Attacks by Randomized [MASK]](https://mlsanity.com/paper/2105.03743/) (2021)
- [How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?](https://mlsanity.com/paper/2112.11668/) (2021)
- [On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex](https://mlsanity.com/paper/2301.12868/) (2023)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrifice](https://mlsanity.com/paper/2105.14553/) (2021)
- [Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing](https://mlsanity.com/paper/2108.04990/) (2021)
- [The Role of Explainability in Assuring Safety of Machine Learning in Healthcare](https://mlsanity.com/paper/2109.00520/) (2021)
- [Adversarial Attacks on Deep Models for Financial Transaction Records](https://mlsanity.com/paper/2106.08361/) (2021)
- [Certified Robustness Against Natural Language Attacks by Causal Intervention](https://mlsanity.com/paper/2205.12331/) (2022)
- [A Survey of Adversarial Defences and Robustness in NLP](https://mlsanity.com/paper/2203.06414/) (2022)
- [Phrase-level Textual Adversarial Attack with Label Preservation](https://mlsanity.com/paper/2205.10710/) (2022)
- [Multi-granularity Textual Adversarial Attack with Behavior Cloning](https://mlsanity.com/paper/2109.04367/) (2021)
- [A Survey on Out-of-Distribution Evaluation of Neural NLP Models](https://mlsanity.com/paper/2306.15261/) (2023)
- [Certified Robustness to Programmable Transformations in LSTMs](https://mlsanity.com/paper/2102.07818/) (2021)
- [Efficient Combinatorial Optimization for Word-level Adversarial Textual Attack](https://mlsanity.com/paper/2109.02229/) (2021)
- [A Strong Baseline for Query Efficient Attacks in a Black Box Setting](https://mlsanity.com/paper/2109.04775/) (2021)
- [The King is Naked: on the Notion of Robustness for Natural Language Processing](https://mlsanity.com/paper/2112.07605/) (2021)
- [AEON: A Method for Automatic Evaluation of NLP Test Cases](https://mlsanity.com/paper/2205.06439/) (2022)
- [Adversarial Robustness for Tabular Data through Cost and Utility Awareness](https://mlsanity.com/paper/2208.13058/) (2022)
- [TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack](https://mlsanity.com/paper/2210.15221/) (2022)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling](https://mlsanity.com/paper/2101.11310/) (2021)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [Robust Textual Embedding against Word-level Adversarial Attacks](https://mlsanity.com/paper/2202.13817/) (2022)

## Abstract (arXiv)

In this paper, we present an effective method to craft text adversarial samples, revealing one important yet underestimated fact that DNN-based text classifiers are also prone to adversarial sample attack. Specifically, confronted with different adversarial scenarios, the text items that are important for classification are identified by computing the cost gradients of the input (white-box attack) or generating a series of occluded test samples (black-box attack). Based on these items, we design three perturbation strategies, namely insertion, modification, and removal, to generate adversarial samples. The experiment results show that the adversarial samples generated by our method can successfully fool both state-of-the-art character-level and word-level DNN-based text classifiers. The adversarial samples can be perturbed to any desirable classes without compromising their utilities. At the same time, the introduced perturbation is difficult to be perceived.