mlsanityData to 14 Oct 2026

AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

arXiv 2609.06972 ↗First posted 7 Sep 2026cs.CR cs.AI cs.LGcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read in order: a benign prefix gives way to actions that serve the attacker rather than the user. Existing benchmarks measure whether such attacks succeed against live agents, and existing guard models judge a trace as a whole; no public corpus labels, step by step, where an injection enters a trajectory and which steps it corrupts. We present AgentDrift, a benchmark of 12,536 synthetic tool-call trajectories over five agent domains in which every one of the 71,024 steps carries one of four labels: benign, injection point, hijacked, or failed injection. The corpus contains 4,000 benign, 5,536 attacked, 1,500 failed-attack, and 1,500 hard-negative trajectories; attacked trajectories follow three compliance patterns whose label strings obey a stated regular grammar. Failed attacks carry an injection the agent resisted, and hard negatives carry legitimate content that resembles an attack, so a detector must separate attempt from success and deviation from novelty. Trajectories were generated by a single open model under category-specific protocols, enforced by a closed-vocabulary structural validator, screened by an LLM judge, and audited by hand on 1,200 trajectories; we show that the LLM judge was itself fooled by the hard negatives. A surface-feature logistic regression recovers only 55.4% of attacks (F1 0.647), including only 8.2% of partial hijacks and 23.1% of delayed executions, so nearly half of the attacks require modeling the behavioral sequence. We measure template concentration, attack-goal-family collapse, and world-identity leakage in the generated data, and release the corpus with its documentation under CC BY 4.0.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 41 of its 52 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

41 of 52 references.

Closest ancestors · 21

Same direction, newest first

  1. 2026StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing25 Aug
  2. 2026Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework7 Aug
  3. 2026TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories8 Apr
  4. 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis2 Apr
  5. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis11 Feb
  6. 2026TrajAD: Trajectory Anomaly Detection for Trustworthy LLM Agents6 Feb
  7. 2026WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents3 Feb
  8. 2026Trajectory Guard -- A Lightweight, Sequence-Aware Model for Real-Time Anomaly Detection in Agentic AI2 Jan
  9. 2025TraceAegis: Securing LLM-Based Agents via Hierarchical and Behavioral Anomaly Detection13 Oct
  10. 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
  11. 2025RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments18 Jun
  12. 2025AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models29 May
9 more
  1. 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
  2. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  3. 2025Defeating Prompt Injections by Design24 Mar
  4. 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
  5. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
  6. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  7. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  8. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
  9. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 20

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  3. 2018Datasheets for Datasetscited by 1,386
  4. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  5. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  6. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  7. 2024ToolACE: Winning the Points of LLM Function Callingcited by 216
  8. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 198
  9. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 185
  10. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  11. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 116
  12. 2024Croissant: A Metadata Format for ML-Ready Datasetscited by 81
8 more
  1. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
  2. 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 59
  3. 2025PromptShield: Deployable Detection for Prompt Injection Attackscited by 50
  4. 2026Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactionscited by 28
  5. 2025Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?cited by 18
  6. 2026PIArena: A Platform for Prompt Injection Evaluationcited by 11
  7. 2026StepShield: When, Not Whether to Intervene on Rogue Agentscited by 7
  8. 2026MCP-38: A Comprehensive Threat Taxonomy for Model Context Protocol Systems (v1.0)cited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
  2. 2026Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
  3. 2026INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems
  4. 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
  5. 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
  6. 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification
  7. 2026BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents
  8. 2026Towards a Unified Misuse Monitoring Benchmark