Reassembling Distributed Risk: Trajectory-Conditioned Action Generation for Multi-Turn Agent Safety
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Tool-using LLM agents extend security risks beyond generated text to actions that affect external systems. Under multi-turn decomposition attacks, a harmful objective can be distributed across individually plausible requests and tool calls, becoming apparent only from the accumulated trajectory. Existing defenses either rely on auxiliary online reasoning to recover long-horizon security evidence or assess actions after generation, often incurring additional inference cost or depending on runtime-specific action representations. We propose Reassembling Distributed Risk (ReDiR), a generation-time defense that conditions action generation on trajectory-level security evidence. Before each action, ReDiR compresses the current trajectory into a compact latent safety representation and injects it into the frozen base model. The representation is learned through same-model, cross-view supervision, where safe behavior from an explicit task view provides supervision for recovering distributed safety evidence from the original multi-turn trajectory. This design enables ReDiR to integrate cross-turn security information directly within the generation process without relying on a separate action-level safety module. We evaluate ReDiR on two agent-safety benchmarks across three model families and eight held-out tool domains. ReDiR reduces attack success rates to below 8%, transfers to unseen tool domains, and preserves benign fidelity with low computational overhead.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 46 of its 55 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
46 of 55 references.
Closest ancestors · 19
Same direction, newest first
- 2026TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safety30 May
- 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory4 May
- 2026A Framework for Formalizing LLM Agent Security19 Mar
- 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents13 Feb
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
- 2025Enforcing Temporal Constraints for LLM Agents25 Dec
- 2025STAC: When Innocent Tools Form Dangerous Chains for LLM Agents30 Sep
- 2025AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models29 May
- 2025SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator23 May
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025SafeArena: Evaluating the Safety of Autonomous Web Agents6 Mar
7 more
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning13 Jun
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 27
- 2023GPT-4 Technical Reportcited by 18,396
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
15 more
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
- 2023Learning to Compress Prompts with Gist Tokenscited by 390
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
- 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
- 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
- 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 54
- 2024Revisiting Knowledge Distillation for Autoregressive Language Modelscited by 42
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
- 2026Enhancing Agent Safety Judgment: Controlled Benchmark Rewriting and Analogical Reasoning for Deceptive Out-of-Distribution Scenarios
- 2026Representation Transitions Reveal Emerging Safety Risks in Multi-Turn LLM Agents
- 2026Toward a Principled Framework for Agent Safety Measurement
- 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
- 2026ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents
- 2026SafeAgent: A Runtime Protection Architecture for Agentic Systems
- 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems