mlsanityData to 14 Oct 2026

Adversarial Decoding: Generating Readable Documents for Adversarial Objectives

arXiv 2410.02163 ↗First posted 3 Oct 2024cs.CL cs.CR cs.LGcited by 16

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

We design, implement, and evaluate adversarial decoding, a new, generic text generation technique that produces readable documents for different adversarial objectives. Prior methods either produce easily detectable gibberish, or cannot handle objectives that include embedding similarity. In particular, they only work for direct attacks (such as jailbreaking) and cannot produce adversarial text for realistic indirect injection, e.g., documents that (1) are retrieved in RAG systems in response to broad classes of queries, and also (2) adversarially influence subsequent generation. We also show that fluency (low perplexity) is not sufficient to evade filtering. We measure the effectiveness of adversarial decoding for different objectives, including RAG poisoning, jailbreaking, and evasion of defensive filters, and demonstrate that it outperforms existing methods while producing readable adversarial documents.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 30 of its 48 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

30 of 48 references.

References · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
18 more
  1. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  4. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  5. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  6. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 264
  7. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  8. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  9. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  10. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 151
  11. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 149
  12. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
  13. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  14. 2020Reevaluating Adversarial Examples in Natural Languagecited by 85
  15. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 65
  16. 2020Adversarial Semantic Collisionscited by 42
  17. 2024Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applicationscited by 33
  18. 2019HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 11.

16 papers.

Sort:

By area:

  1. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 40
  2. 2026Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registrycited by 12
  3. 2025Universal Zero-shot Embedding Inversioncited by 8
  4. 2025Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreakscited by 6
  5. 2025Uncovering Competing Poisoning Attacks in Retrieval-Augmented Generationcited by 5
  6. 2026Deep-Research Agents Can Be Poisoned via User-Generated Contentcited by 4
  7. 2025DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)cited by 2
  8. 2026MemPot: Defending Against Memory Extraction Attack with Optimized Honeypotscited by 2
  9. 2026Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidencecited by 1
  10. 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoningcited by 1
  11. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generationcited by 1
  12. 2026Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoningcited by 1
4 more
  1. 2026RAGSentinel: Certifiable Geometric Consensus for Robust Retrieval-Augmented Generationcited by 0
  2. 2026ViTeGate: Visual-Textual Triggered Knowledge Poisoning for Vision-Language Retrieval-Augmented Generationcited by 0
  3. 2026Detecting and Localizing Segment-Level Poisoning in Multi-Source LLM-Agent Inputscited by 0
  4. 2026Divide and Doubt: Diverse Distributed Poisoning for Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks
  2. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  3. 2026CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents
  4. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models
  5. 2026Towards Unsupervised Adversarial Document Detection in Retrieval Augmented Generation Systems
  6. 2025AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Prompt
  7. 2026RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning
  8. 2025DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injection