Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address this threat, little is known about the internals of agentic LLMs when they are exposed to IPI attacks. For simplicity, we refer to this condition as IPI exposure. In this paper, we study IPI exposure from three perspectives. (1) Probing: Across eight models, including the 753B-parameter GLM-5.2 and the 2.8T-parameter Kimi-K3, simple linear probes trained on pre-generation hidden states can predict LLMs' IPI exposure. These probes achieve 0.90+ AUROC on unseen attacks, agent instructions, and task suites; they remain robustly predictive under adaptive attacks and in cross-lingual settings. (2) Defense: We reveal and diagnose a knowledge-action gap: post-trained LLMs encode signals predictive of IPI exposure, yet do not reliably bind these signals to safe agentic actions. We therefore introduce a probe-gated reasoning-based defense to bridge this gap at test time. On difficult AgentDojo settings, it substantially reduces attack success rate, e.g., from 34.6% to 0% on Qwen3.5-27B, and better preserves clean-task utility than the baselines. (3) Explanation: We introduce an analysis framework that identifies natural-language explanations strongly correlated with probe-captured signals. The resulting profiles differ across models: latent signals can align with either direct IPI-exposure sensing or indirect operational cues. Code is available at https://github.com/jianshuod/IPI-exposure-signal.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 83 of its 97 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
83 of 97 references.
Closest ancestors · 31
Same direction, newest first
- 2026PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis14 Jul
- 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents22 Jun
- 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment15 Apr
- 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs4 Apr
- 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition16 Mar
- 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution8 Feb
- 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit9 Jan
- 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs9 Dec
- 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?23 Oct
- 2025TraceAegis: Securing LLM-Based Agents via Hierarchical and Behavioral Anomaly Detection13 Oct
- 2025SafeSearch: Automated Red-Teaming of LLM-Based Search Agents28 Sep
19 more
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
- 2025HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimization6 Aug
- 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
- 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
- 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction5 Mar
- 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
- 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks12 Feb
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
- 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning13 Jun
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
Other references · 52
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2025s1: Simple test-time scalingcited by 1,341
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
38 more
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
- 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
- 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
- 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
- 2025Measuring AI Ability to Complete Long Software Taskscited by 156
- 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
- 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 129
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
- 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 95
- 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
- 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
- 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 65
- 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 59
- 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
- 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 52
- 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
- 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 41
- 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thoughtcited by 40
- 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 32
- 2025PromptLocate: Localizing Prompt Injection Attackscited by 28
- 2024ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Modelscited by 25
- 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
- 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Featurescited by 20
- 2026Internal Representations as Indicators of Hallucinations in Agent Tool Selectioncited by 19
- 2026Prompt Injection as Role Confusioncited by 17
- 2024Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applicationscited by 16
- 2025Thought Branches: Interpreting LLM Reasoning Requires Resamplingcited by 16
- 2025Defense against Prompt Injection Attacks via Mixture of Encodingscited by 14
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Readable Before Actionable: Causal Tracing of Indirect Prompt Injectioncited by 1
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agentscited by 0
- 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steeringcited by 0
- 2026False Floors: LLM Safety Routing Evaluations Break Under Distribution Shiftcited by 0
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Followcited by 0
- 2026Prompt-Injection Probes Learn the Conversation, Not Just the Injectioncited by 0
- 2026Are You Reading the Right Thing? Rethinking Internal Signals for Prompt-Injection Detectioncited by 0
- 2026Are You Reading the Right Thing? Rethinking Internal Signals for Prompt-Injection Detection14 Oct 2026
- 2026Prompt-Injection Probes Learn the Conversation, Not Just the Injection13 Oct 2026
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow12 Oct 2026
- 2026Readable Before Actionable: Causal Tracing of Indirect Prompt Injection4 Oct 2026
- 2026False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift1 Oct 2026
- 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering29 Sep 2026
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents9 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild
- 2026Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents
- 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
- 2026Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
- 2026AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents
- 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
- 2026APEX: Active Protection at Execution Boundaries for LLM Agents