mlsanityData to 7 Oct 2026

Representation Transitions Reveal Emerging Safety Risks in Multi-Turn LLM Agents

arXiv 2610.00400 ↗First posted 30 Sep 2026cs.LG cs.AI cs.CRcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or states in isolation. We show that such attacks leave a detectable signature in the agent's internal representations: harmful behavior emerges as an accumulated representation transition across context updates, whose triggering context can be identified from the same signal. We further find that naive aggregation is confounded by benign representation drift, as a contrastive safety direction need not assign zero to benign transitions. We address this by denoising the direction, anchoring benign traffic at zero and removing its leading variation directions, with no runtime cost. These findings motivate DART, a runtime framework that detects and attributes representation shifts and intervenes with targeted reminders. Across six models and two multi-turn benchmarks, DART reduces attack success from 84% to 25% on MT-AgentRisk, catching every attack at a mean false-alarm rate of 12%, and from 97% to 52% on ASEval, at costs in benign non-refusal of 8% and 0%, respectively. On MT-AgentRisk, it outperforms ToolShield, the state-of-the-art multi-turn defense, on all six models: under the same protocol, ToolShield reaches only 55%. Denoising is critical: on ASEval, the undenoised monitor catches only 7%-40% of attacks, while the denoised monitor catches 60%-85%. The same monitor covers single-turn indirect injection without modification and adds only 0.14-0.56 s overhead per monitored step without requiring an auxiliary model, making it a lightweight complement to computation-heavy speculative defenses.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 31 of its 39 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

31 of 39 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling26 May
  2. 2026ASEval: Automated Trajectory-Level Security Testing for Autonomous Agents21 May
  3. 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents13 Feb
  4. 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs9 Dec
  5. 2025ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction1 Aug
  6. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  7. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
  8. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
  9. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  10. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning13 Jun
  11. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  12. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
1 more
  1. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  11. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  12. 2019Designing and Interpreting Probes with Control Taskscited by 608
6 more
  1. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  2. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  3. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
  4. 2025Qwen3Guard Technical Reportcited by 125
  5. 2024Transformers need glasses! Information over-squashing in language taskscited by 87
  6. 2025RvLLM: LLM Runtime Verification with Domain Knowledgecited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States
  2. 2026The Geometry of Harmfulness in Multi-Turn Attacks
  3. 2026Toward a Principled Framework for Agent Safety Measurement
  4. 2026HARP: Measuring Harm Amplification in Multi-Agent LLM Systems
  5. 2026Reassembling Distributed Risk: Trajectory-Conditioned Action Generation for Multi-Turn Agent Safety
  6. 2026ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems
  7. 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
  8. 2026Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback