Beyond Over-Refusal: Defending Indirect Prompt Injection via Latent Instruction Manifolds
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Large Language Models (LLMs) have been integrated into complex ecosystems (e.g., Code Agents), while Indirect Prompt Injection (IPI) attacks have emerged as critical barriers to their safe deployment. Attackers exploit LLMs' indistinguishability between "instructions" and "data" to manipulate LLMs via maliciously injected instructions. Existing defenses, however, face an intractable safety-utility trade-off: most guardrails either incur high latency or suffer from severe over-refusal. In this paper, we first demonstrate that LLMs can separate instruction from data intrinsically with both theoretical and empirical evidence. Inspired by this insight, we propose AEGIS (Adaptive Ensemble Guard for Injection Shielding). AEGIS extracts instruction-sensitive projectors to identify malicious instructions and leverages a Unified Multi-Layer Consensus mechanism that aggregates topologically distinct signals across the network depth. Empirical evaluations show that AEGIS achieves remarkable detection performance against both heuristic and optimization-based attacks compared to baselines, highlighting its potential to mitigate IPI. Code is available at https://github.com/xaddwell/AEGIS
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 24 of its 50 references that are in the map. First posted on arXiv 23 Aug 2026.
Builds on
24 of 50 references.
Closest ancestors · 17
Same direction, newest first
- 2026SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agents28 Apr
- 2025PISanitizer: Preventing Prompt Injection to Long-Context LLMs via Prompt Sanitization13 Nov
- 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features15 Oct
- 2025PromptLocate: Localizing Prompt Injection Attacks14 Oct
- 2025Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling9 Sep
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
- 2025Defending Against Prompt Injection With a Few DefensiveTokens10 Jul
- 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks15 Apr
- 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?23 Feb
- 2024Defense Against Prompt Injection Attack by Leveraging Attack Techniques1 Nov
- 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMs1 Nov
5 more
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models30 Oct
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
- 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks6 Mar
- 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
- 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuning29 Dec
Other references · 7
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023Detecting Language Model Attacks with Perplexitycited by 431
- 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 81
- 2025Leveraging the true depth of LLMscited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Defending against Indirect Prompt Injection by Instruction Detection
- 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection
- 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification
- 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis
- 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
- 2026BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes
- 2026Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learning
- 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior