# A Constraint-Enforcing Reward for Adversarial Attacks on Text Classifiers

arXiv: https://arxiv.org/abs/2405.11904 · HTML: https://mlsanity.com/paper/2405.11904/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control](https://mlsanity.com/paper/1611.02796/) (2016)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples](https://mlsanity.com/paper/1803.01128/) (2018)
- [Reevaluating Adversarial Examples in Natural Language](https://mlsanity.com/paper/2004.14174/) (2020)
- [Imitation Attacks and Defenses for Black-box Machine Translation Systems](https://mlsanity.com/paper/2004.15015/) (2020)
- [On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models](https://mlsanity.com/paper/1903.06620/) (2019)
- [Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples](https://mlsanity.com/paper/2009.06368/) (2020)
- [Generating Natural Language Adversarial Examples on a Large Scale with Generative Models](https://mlsanity.com/paper/2003.10388/) (2020)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model](https://mlsanity.com/paper/1909.07873/) (2019)
- [DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation](https://mlsanity.com/paper/1712.05419/) (2017)

## Built on by (most cited first, 2 of 2)

- [Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.06837/) (2025)
- [SMAB: MAB based word Sensitivity Estimation Framework and its Applications in Adversarial Text Generation](https://mlsanity.com/paper/2502.07101/) (2025)

## Abstract (arXiv)

Text classifiers are vulnerable to adversarial examples -- correctly-classified examples that are deliberately transformed to be misclassified while satisfying acceptability constraints. The conventional approach to finding adversarial examples is to define and solve a combinatorial optimisation problem over a space of allowable transformations. While effective, this approach is slow and limited by the choice of transformations. An alternate approach is to directly generate adversarial examples by fine-tuning a pre-trained language model, as is commonly done for other text-to-text tasks. This approach promises to be much quicker and more expressive, but is relatively unexplored. For this reason, in this work we train an encoder-decoder paraphrase model to generate a diverse range of adversarial examples. For training, we adopt a reinforcement learning algorithm and propose a constraint-enforcing reward that promotes the generation of valid adversarial examples. Experimental results over two text classification datasets show that our model has achieved a higher success rate than the original paraphrase model, and overall has proved more effective than other competitive attacks. Finally, we show how key design choices impact the generated examples and discuss the strengths and weaknesses of the proposed approach.