# Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework

arXiv: https://arxiv.org/abs/2110.15317 · HTML: https://mlsanity.com/paper/2110.15317/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 58 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Gradient-based Adversarial Attacks against Text Transformers](https://mlsanity.com/paper/2104.13733/) (2021)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)

## Built on by (most cited first, 28 of 28)

- [DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback](https://mlsanity.com/paper/2311.10081/) (2023)
- [Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP](https://mlsanity.com/paper/2210.10683/) (2022)
- [Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations](https://mlsanity.com/paper/2306.04618/) (2023)
- [Safety of Multimodal Large Language Models on Images and Texts](https://mlsanity.com/paper/2402.00357/) (2024)
- [Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts](https://mlsanity.com/paper/2112.08348/) (2021)
- [A Close Look into the Calibration of Pre-trained Language Models](https://mlsanity.com/paper/2211.00151/) (2022)
- [JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation](https://mlsanity.com/paper/2502.07557/) (2025)
- [Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models](https://mlsanity.com/paper/2309.04461/) (2023)
- [LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses](https://mlsanity.com/paper/2406.04755/) (2024)
- [Impact of Adversarial Training on Robustness and Generalizability of Language Models](https://mlsanity.com/paper/2211.05523/) (2022)
- [Revealing Vulnerabilities in Stable Diffusion via Targeted Attacks](https://mlsanity.com/paper/2401.08725/) (2024)
- [Fluent dreaming for language models](https://mlsanity.com/paper/2402.01702/) (2024)
- [A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation](https://mlsanity.com/paper/2308.15246/) (2023)
- [Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens](https://mlsanity.com/paper/2406.13294/) (2024)
- [AMA-LSTM: Pioneering Robust and Fair Financial Audio Analysis for Stock Volatility Prediction](https://mlsanity.com/paper/2407.18324/) (2024)
- [Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks](https://mlsanity.com/paper/2302.05120/) (2023)
- [From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework](https://mlsanity.com/paper/2305.18503/) (2023)
- [CodeCipher: Learning to Obfuscate Source Code Against LLMs](https://mlsanity.com/paper/2410.05797/) (2024)
- [IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method](https://mlsanity.com/paper/2505.06889/) (2025)
- [Group-based Robustness: A General Framework for Customized Robustness in the Real World](https://mlsanity.com/paper/2306.16614/) (2023)
- [Exploiting Class Probabilities for Black-box Sentence-level Attacks](https://mlsanity.com/paper/2402.02695/) (2024)
- [Pointing out the Shortcomings of Relation Extraction Models with Semantically Motivated Adversarials](https://mlsanity.com/paper/2402.19076/) (2024)
- [DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising](https://mlsanity.com/paper/2407.00248/) (2024)
- [Making Pre-trained Language Models both Task-solvers and Self-calibrators](https://mlsanity.com/paper/2307.11316/) (2023)
- [A Generative Adversarial Attack for Multilingual Text Classifiers](https://mlsanity.com/paper/2401.08255/) (2024)
- [NMT-Obfuscator Attack: Ignore a sentence in translation with only one word](https://mlsanity.com/paper/2411.12473/) (2024)
- [SurvAttack: Black-Box Attack On Survival Models through Ontology-Informed EHR Perturbation](https://mlsanity.com/paper/2412.18706/) (2024)
- [Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text](https://mlsanity.com/paper/2606.27215/) (2026)

## Abstract (arXiv)

Despite recent success on various tasks, deep learning techniques still perform poorly on adversarial examples with small perturbations. While optimization-based methods for adversarial attacks are well-explored in the field of computer vision, it is impractical to directly apply them in natural language processing due to the discrete nature of the text. To address the problem, we propose a unified framework to extend the existing optimization-based adversarial attack methods in the vision domain to craft textual adversarial samples. In this framework, continuously optimized perturbations are added to the embedding layer and amplified in the forward propagation process. Then the final perturbed latent representations are decoded with a masked language model head to obtain potential adversarial samples. In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD). We find our algorithm effective even using proxy gradient information. Therefore, we perform the more challenging transfer black-box attack and conduct comprehensive experiments to evaluate our attack algorithm with several models on three benchmark datasets. Experimental results demonstrate that our method achieves overall better performance and produces more fluent and grammatical adversarial samples compared to strong baseline methods. The code and data are available at urlhttps://github.com/Phantivia/T-PGD.