mlsanityData to 7 Oct 2026

TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling

arXiv 2605.27690 ↗First posted 26 May 2026cs.CL cs.LGcited by 5

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM agents increasingly operate through multi-turn tool use and environment interaction, where safety risks often emerge from intermediate steps long before they surface in the final outcome. Reactive auditing is therefore insufficient: post-hoc diagnosis frequently misses the chance to flag risks while they are unfolding. We propose TRACES, a representation-based proactive auditor that learns prefix-level trajectory risk states from the hidden representations of an observer LLM. TRACES induces latent mechanism features from step representations and models their temporal evolution to estimate whether a partial trajectory is drifting toward unsafe behavior. To sidestep the cost and ambiguity of step-level risk annotation, TRACES is trained with weak trajectory-level supervision while still producing dense prefix-level risk estimates. Across multiple agent safety benchmarks, TRACES improves both full-trajectory safety prediction and proactive risk discrimination. Our analyses further suggest that these risk states can help train a safer agent, highlighting the broader potential of proactive auditing for long-horizon agent safety.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 19 of its 32 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

19 of 32 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark15 Apr
  2. 2026AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents3 Apr
  3. 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis2 Apr
  4. 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents13 Feb
  5. 2026Towards Verifiably Safe Tool Use for LLM Agents12 Jan
  6. 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents31 May
  7. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  8. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
  9. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb

Other references · 10

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  3. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  4. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  5. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Surveycited by 164
  6. 2024GTA: A Benchmark for General Tool Agentscited by 77
  7. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  8. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  9. 2025Representation Bending for Large Language Model Safetycited by 28
  10. 2025Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial Statescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents
  2. 2026AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories
  3. 2026TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safety
  4. 2026AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems
  5. 2025ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
  6. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
  7. 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
  8. 2026Automata from Agent Traces: Failure and Next-Step Prediction