# R&R: Metric-guided Adversarial Sentence Generation

arXiv: https://arxiv.org/abs/2104.08453 · HTML: https://mlsanity.com/paper/2104.08453/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [Certified Robustness to Adversarial Word Substitutions](https://mlsanity.com/paper/1909.00986/) (2019)
- [Distributionally Robust Language Modeling](https://mlsanity.com/paper/1909.02060/) (2019)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Contextualized Perturbation for Textual Adversarial Attack](https://mlsanity.com/paper/2009.07502/) (2020)
- [Deep Text Classification Can be Fooled](https://mlsanity.com/paper/1704.08006/) (2017)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)
- [OpenAttack: An Open-source Textual Adversarial Attack Toolkit](https://mlsanity.com/paper/2009.09191/) (2020)
- [Fake News Detection via NLP is Vulnerable to Adversarial Attacks](https://mlsanity.com/paper/1901.09657/) (2019)
- [Satirical News Detection and Analysis using Attention Mechanism and Linguistic Features](https://mlsanity.com/paper/1709.01189/) (2017)
- [SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher](https://mlsanity.com/paper/2011.08908/) (2020)
- [Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model](https://mlsanity.com/paper/1909.07873/) (2019)
- [DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation](https://mlsanity.com/paper/1712.05419/) (2017)

## Built on by (most cited first, 6 of 6)

- [VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models](https://mlsanity.com/paper/2310.04655/) (2023)
- [VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models](https://mlsanity.com/paper/2402.11083/) (2024)
- [Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation](https://mlsanity.com/paper/2410.09318/) (2024)
- [Adversarial Attacks on VQA-NLE: Exposing and Alleviating Inconsistencies in Visual Question Answering Explanations](https://mlsanity.com/paper/2508.12430/) (2025)
- [MultiPhishGuard: An Explainable and Adaptive Multi-Agent LLM System for Phishing Email Detection](https://mlsanity.com/paper/2505.23803/) (2025)
- [Coordinated Robustness Evaluation Framework for Vision-Language Models](https://mlsanity.com/paper/2506.05429/) (2025)

## Abstract (arXiv)

Adversarial examples are helpful for analyzing and improving the robustness of text classifiers. Generating high-quality adversarial examples is a challenging task as it requires generating fluent adversarial sentences that are semantically similar to the original sentences and preserve the original labels, while causing the classifier to misclassify them. Existing methods prioritize misclassification by maximizing each perturbation's effectiveness at misleading a text classifier; thus, the generated adversarial examples fall short in terms of fluency and similarity. In this paper, we propose a rewrite and rollback (R&R) framework for adversarial attack. It improves the quality of adversarial examples by optimizing a critique score which combines the fluency, similarity, and misclassification metrics. R&R generates high-quality adversarial examples by allowing exploration of perturbations that do not have immediate impact on the misclassification metric but can improve fluency and similarity metrics. We evaluate our method on 5 representative datasets and 3 classifier architectures. Our method outperforms current state-of-the-art in attack success rate by +16.2%, +12.8%, and +14.0% on the classifiers respectively. Code is available at https://github.com/DAI-Lab/fibber