mlsanityData to 7 Oct 2026

PIArena: A Platform for Prompt Injection Evaluation

arXiv 2604.08499 ↗First posted 9 Apr 2026cs.CR cs.AI cs.CLcited by 10

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Prompt injection attacks pose serious security risks across a wide range of real-world applications. While receiving increasing attention, the community faces a critical gap: the lack of a unified platform for prompt injection evaluation. This makes it challenging to reliably compare defenses, understand their true robustness under diverse attacks, or assess how well they generalize across tasks and benchmarks. For instance, many defenses initially reported as effective were later found to exhibit limited robustness on diverse datasets and attacks. To bridge this gap, we introduce PIArena, a unified and extensible platform for prompt injection evaluation that enables users to easily integrate state-of-the-art attacks and defenses and evaluate them across a variety of existing and new benchmarks. We also design a dynamic strategy-based attack that adaptively optimizes injected prompts based on defense feedback. Through comprehensive evaluation using PIArena, we uncover critical limitations of state-of-the-art defenses: limited generalizability across tasks, vulnerability to adaptive attacks, and fundamental challenges when an injected task aligns with the target task. The code and datasets are available at https://github.com/sleeepeer/PIArena.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 52 of its 79 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

52 of 79 references.

Closest ancestors · 21

Same direction, newest first

  1. 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses13 Mar
  2. 2025PISanitizer: Preventing Prompt Injection to Long-Context LLMs via Prompt Sanitization13 Nov
  3. 2025Defending Against Prompt Injection with DataFilter22 Oct
  4. 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features15 Oct
  5. 2025RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection6 Oct
  6. 2025SecInfer: Preventing Prompt Injection via Inference-time Scaling29 Sep
  7. 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attacks10 Jul
  8. 2025A Critical Evaluation of Defenses against Prompt Injection Attacks23 May
  9. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  10. 2025Progent: Securing AI Agents with Privilege Control16 Apr
  11. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks15 Apr
  12. 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models30 Oct
9 more
  1. 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
  2. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  3. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective27 Sep
  4. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions19 Apr
  5. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks6 Mar
  6. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts26 Feb
  7. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
  8. 2023Prompt Injection attack against LLM-integrated Applications8 Jun
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  11. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
19 more
  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  3. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  4. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  5. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  6. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  7. 2021Efficient Attentions for Long Document Summarizationcited by 342
  8. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  9. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  10. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  11. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  12. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  13. 2023LongCoder: A Long-Range Pre-trained Language Model for Code Completioncited by 110
  14. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  15. 2024Attacking Large Language Models with Projected Gradient Descentcited by 91
  16. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agentscited by 81
  17. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  18. 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
  19. 2025Black-box Optimization of LLM Outputs by Asking for Directionscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  2. 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectoriescited by 2
  3. 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
  4. 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agentscited by 1
  5. 2026FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruptioncited by 0
  6. 2026ContainmentBench: Trace-Based Evaluation of Post-Exposure Containment in Tool-Using LLM Agentscited by 0
  7. 2026Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectorscited by 0
  8. 2026Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learningcited by 0
  9. 2026CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agentscited by 0
  10. 2026Secure Speculative Decoding for Large Language Modelscited by 0