mlsanityData to 14 Oct 2026

FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption

arXiv 2604.28157 ↗First posted 30 Apr 2026cs.CRcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

Long-context large language models (LLMs)-for example, Gemini-3.1-Pro and Qwen-3.5-are widely used to empower many real-world applications, such as retrieval-augmented generation, autonomous agents, and AI assistants. However, security remains a major concern for their widespread deployment, with threats such as prompt injection and knowledge corruption. To quantify the security risks faced by LLMs under these threats, the research community has developed heuristic-based and optimization-based red-teaming methods. Optimization-based methods generally produce stronger attacks than heuristic attacks and thus provide a more rigorous assessment of LLM security risks. However, they are often resource-intensive, requiring significant computation and GPU memory, especially for long context scenarios. The resource-intensive nature poses a major obstacle for the community (especially academic researchers) to systematically evaluate the security risks of long-context LLMs and assess the effectiveness of defense strategies at scale. In this work, we propose FlashRT, the first framework to improve the efficiency (in terms of both computation and memory) for optimization-based prompt injection and knowledge corruption attacks under long-context LLMs. Through extensive evaluations, we find that FlashRT consistently delivers a 2x-7x speedup (e.g., reducing runtime from one hour to less than ten minutes) and a 2x-4x reduction in GPU memory consumption (e.g., reducing from 264.1 GB to 65.7 GB GPU memory for a 32K token context) compared to state-of-the-art baseline nanoGCG. FlashRT can be broadly applied to black-box optimization methods, such as TAP and AutoDAN. We hope FlashRT can serve as a red-teaming tool to enable systematic evaluation of long-context LLM security. The code is available at: https://github.com/Wang-Yanting/FlashRT

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 49 of its 68 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

49 of 68 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models3 Feb
  2. 2025GraphRAG under Fire23 Jan
  3. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun
  4. 2024Certifiably Robust RAG against Retrieval Corruption24 May
  5. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models22 May
  6. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb

Other references · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
31 more
  1. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  2. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  3. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  5. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  6. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  7. 2022Efficiently Scaling Transformer Inferencecited by 539
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  9. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
  10. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  11. 2021Efficient Attentions for Long Document Summarizationcited by 345
  12. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
  13. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
  14. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
  15. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  16. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
  17. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  18. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
  19. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
  20. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  21. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 50
  22. 2024Long Code Arena: a Set of Benchmarks for Long-Context Code Modelscited by 50
  23. 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 43
  24. 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 36
  25. 2025RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injectioncited by 36
  26. 2025OpenAI's Approach to External Red Teaming for AI Models and Systemscited by 28
  27. 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 16
  28. 2025Learning to Attribute with Attentioncited by 13
  29. 2026PIArena: A Platform for Prompt Injection Evaluationcited by 11
  30. 2025Black-box Optimization of LLM Outputs by Asking for Directionscited by 7
  31. 2026TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs
  2. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations
  3. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)
  4. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Models
  5. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
  6. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models
  7. 2024Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding
  8. 2025Prompt Optimization and Evaluation for LLM Automated Red Teaming