# Generating Valid and Natural Adversarial Examples with Large Language Models

arXiv: https://arxiv.org/abs/2311.11861 · HTML: https://mlsanity.com/paper/2311.11861/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 20 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP](https://mlsanity.com/paper/2210.10683/) (2022)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [OpenAttack: An Open-source Textual Adversarial Attack Toolkit](https://mlsanity.com/paper/2009.09191/) (2020)
- [Towards Improving Adversarial Training of NLP Models](https://mlsanity.com/paper/2109.00544/) (2021)
- [ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark](https://mlsanity.com/paper/2303.13648/) (2023)
- [MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction](https://mlsanity.com/paper/2211.07342/) (2022)
- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks](https://mlsanity.com/paper/2305.15587/) (2023)
- [Modeling Adversarial Attack on Pre-trained Language Models as Sequential Decision Making](https://mlsanity.com/paper/2305.17440/) (2023)
- [Are Synonym Substitution Attacks Really Synonym Substitution Attacks?](https://mlsanity.com/paper/2210.02844/) (2022)

## Built on by (most cited first, 12 of 12)

- [Metamorphic Testing of Large Language Models for Natural Language Processing](https://mlsanity.com/paper/2511.02108/) (2025)
- [Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection](https://mlsanity.com/paper/2410.15623/) (2024)
- [Thinker-DDM: Modeling Deliberation for Machine Translation with a Drift-Diffusion Process](https://mlsanity.com/paper/2402.10699/) (2024)
- [Bounded PCTL Model Checking of Large Language Model Outputs](https://mlsanity.com/paper/2509.18836/) (2025)
- [Red-teaming Activation Probes using Prompted LLMs](https://mlsanity.com/paper/2511.00554/) (2025)
- [CryptoQA: A Large-scale Question-answering Dataset for AI-assisted Cryptography](https://mlsanity.com/paper/2512.02625/) (2025)
- [Automatic Replication of LLM Mistakes in Medical Conversations](https://mlsanity.com/paper/2512.20983/) (2025)
- [When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents](https://mlsanity.com/paper/2605.24069/) (2026)
- [Advancing NLP Security by Leveraging LLMs as Adversarial Engines](https://mlsanity.com/paper/2410.18215/) (2024)
- [CogniAlign: Survivability-Grounded Multi-Agent Moral Reasoning for Safe and Transparent AI](https://mlsanity.com/paper/2509.13356/) (2025)
- [RAG-targeted Adversarial Attack on LLM-based Threat Detection and Mitigation Framework](https://mlsanity.com/paper/2511.06212/) (2025)
- [S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP](https://mlsanity.com/paper/2606.13439/) (2026)

## Abstract (arXiv)

Deep learning-based natural language processing (NLP) models, particularly pre-trained language models (PLMs), have been revealed to be vulnerable to adversarial attacks. However, the adversarial examples generated by many mainstream word-level adversarial attack models are neither valid nor natural, leading to the loss of semantic maintenance, grammaticality, and human imperceptibility. Based on the exceptional capacity of language understanding and generation of large language models (LLMs), we propose LLM-Attack, which aims at generating both valid and natural adversarial examples with LLMs. The method consists of two stages: word importance ranking (which searches for the most vulnerable words) and word synonym replacement (which substitutes them with their synonyms obtained from LLMs). Experimental results on the Movie Review (MR), IMDB, and Yelp Review Polarity datasets against the baseline adversarial attack models illustrate the effectiveness of LLM-Attack, and it outperforms the baselines in human and GPT-4 evaluation by a significant margin. The model can generate adversarial examples that are typically valid and natural, with the preservation of semantic meaning, grammaticality, and human imperceptibility.