mlsanityData to 14 Oct 2026

Defense against Prompt Injection Attacks via Mixture of Encodings

arXiv 2504.07467 ↗First posted 10 Apr 2025cs.CLcited by 14

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Large Language Models (LLMs) have emerged as a dominant approach for a wide range of NLP tasks, with their access to external information further enhancing their capabilities. However, this introduces new vulnerabilities, known as prompt injection attacks, where external content embeds malicious instructions that manipulate the LLM's output. Recently, the Base64 defense has been recognized as one of the most effective methods for reducing success rate of prompt injection attacks. Despite its efficacy, this method can degrade LLM performance on certain NLP tasks. To address this challenge, we propose a novel defense mechanism: mixture of encodings, which utilizes multiple character encodings, including Base64. Extensive experimental results show that our method achieves one of the lowest attack success rates under prompt injection attacks, while maintaining high performance across all NLP tasks, outperforming existing character encoding-based defense methods. This underscores the effectiveness of our mixture of encodings strategy for both safety and task performance metrics.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 27 of its 33 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

27 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions19 Apr
  2. 2024Defending Against Indirect Prompt Injection Attacks With Spotlighting20 Mar
  3. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models21 Dec
  4. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game2 Nov
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  11. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
10 more
  1. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  2. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  3. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  4. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  5. 2023Detecting Language Model Attacks with Perplexitycited by 431
  6. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  7. 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 70
  8. 2022PromptBoosting: Black-Box Text Classification with Ten Forward Passescited by 36
  9. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 33
  10. 2023PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refinecited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

  1. 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystemcited by 19
  2. 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Managementcited by 10
  3. 2025PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariancecited by 8
  4. 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
  5. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 7
  6. 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skillscited by 5
  7. 2026Conjunctive Prompt Attacks in Multi-Agent LLM Systemscited by 2
  8. 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safetycited by 2
  9. 2025SteganoBackdoor: Evading Data-Poisoning Defenses via Steganographic Backdoorscited by 1
  10. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  11. 2025Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMscited by 0
  12. 2026Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detectioncited by 0
2 more
  1. 2026SafeMed-R1: Clinician-Audited Safety and Ethics Alignment for Medical Large Language Modelscited by 0
  2. 2026Origin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary Separationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Automatic and Universal Prompt Injection Attacks against Large Language Models
  2. 2024Defense Against Prompt Injection Attack by Leveraging Attack Techniques
  3. 2025A Critical Evaluation of Defenses against Prompt Injection Attacks
  4. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction
  5. 2025Prompt Injection as an Emerging Threat: Evaluating the Resilience of Large Language Models
  6. 2025Attack and defense techniques in large language models: A survey and new perspectives
  7. 2026An Empirical Evaluation of Prompt Injection Vulnerabilities in Large Language Models Across Multilingual and Obfuscated Attack Scenarios
  8. 2023Prompt Injection attack against LLM-integrated Applications