# TraceAegis: Securing LLM-Based Agents via Hierarchical and Behavioral Anomaly Detection arXiv: https://arxiv.org/abs/2510.11203 ยท HTML: https://mlsanity.com/paper/2510.11203/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 20 of its 33 references that are in the map. - First posted on arXiv 13 Oct 2025. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) ## Builds on (most cited first, 20 of 20) - [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023) - [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024) - [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024) - [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024) - [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025) - [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024) - [ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning](https://mlsanity.com/paper/2503.22738/) (2025) - [DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks](https://mlsanity.com/paper/2504.11358/) (2025) - [Progent: Securing AI Agents with Privilege Control](https://mlsanity.com/paper/2504.11703/) (2025) - [PromptArmor: Simple yet Effective Prompt Injection Defenses](https://mlsanity.com/paper/2507.15219/) (2025) - [Jatmo: Prompt Injection Defense by Task-Specific Finetuning](https://mlsanity.com/paper/2312.17673/) (2023) - [The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2412.16682/) (2024) - [System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective](https://mlsanity.com/paper/2409.19091/) (2024) - [Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents](https://mlsanity.com/paper/2503.15547/) (2025) - [AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection](https://mlsanity.com/paper/2508.01249/) (2025) - [Defense Against Prompt Injection Attack by Leveraging Attack Techniques](https://mlsanity.com/paper/2411.00459/) (2024) - [AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents](https://mlsanity.com/paper/2509.07764/) (2025) - [When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs](https://mlsanity.com/paper/2410.14569/) (2024) - [AgentSight: System-Level Observability for AI Agents Using eBPF](https://mlsanity.com/paper/2508.02736/) (2025) - [Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction](https://mlsanity.com/paper/2504.20472/) (2025) ## Built on by (most cited first, 12 of 12) - [AIR: Improving Agent Safety through Incident Response](https://mlsanity.com/paper/2602.11749/) (2026) - [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026) - [Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security](https://mlsanity.com/paper/2606.11671/) (2026) - [Automata from Agent Traces: Failure and Next-Step Prediction](https://mlsanity.com/paper/2608.23670/) (2026) - [Forensic Trajectory Signatures for Agent Memory Poisoning Detection](https://mlsanity.com/paper/2606.30566/) (2026) - [AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories](https://mlsanity.com/paper/2609.06972/) (2026) - [Content-Aware Attack Detection in LLM Agent Tool-Call Traffic: An Empirical Study of Features, Architectures, and Evaluation Protocols](https://mlsanity.com/paper/2605.11053/) (2026) - [Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications](https://mlsanity.com/paper/2605.24883/) (2026) - [Hybrid Analysis for Secure MCP Tool Use in LLM Agents](https://mlsanity.com/paper/2607.25297/) (2026) - [Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling](https://mlsanity.com/paper/2609.06835/) (2026) - [DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents](https://mlsanity.com/paper/2609.10892/) (2026) - [AgentSpy: Making AI Agent Behavior Observable](https://mlsanity.com/paper/2610.06001/) (2026) ## Abstract (arXiv) LLM-based agents have demonstrated promising adaptability in real-world applications. However, these agents remain vulnerable to a wide range of attacks, such as tool poisoning and malicious instructions, that compromise their execution flow and can lead to serious consequences like data breaches and financial loss. Existing studies typically attempt to mitigate such anomalies by predefining specific rules and enforcing them at runtime to enhance safety. Yet, designing comprehensive rules is difficult, requiring extensive manual effort and still leaving gaps that result in false negatives. As agent systems evolve into complex software systems, we take inspiration from software system security and propose TraceAegis, a provenance-based analysis framework that leverages agent execution traces to detect potential anomalies. In particular, TraceAegis constructs a hierarchical structure to abstract stable execution units that characterize normal agent behaviors. These units are then summarized into constrained behavioral rules that specify the conditions necessary to complete a task. By validating execution traces against both hierarchical and behavioral constraints, TraceAegis is able to effectively detect abnormal behaviors. To evaluate the effectiveness of TraceAegis, we introduce TraceAegis-Bench, a dataset covering two representative scenarios: healthcare and corporate procurement. Each scenario includes 1,300 benign behaviors and 300 abnormal behaviors, where the anomalies either violate the agent's execution order or break the semantic consistency of its execution sequence. Experimental results demonstrate that TraceAegis achieves strong performance on TraceAegis-Bench, successfully identifying the majority of abnormal behaviors.