Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
We introduce Semantic Intent Fragmentation (SIF), an attack class against LLM orchestration systems where a single, legitimately phrased request causes an orchestrator to decompose a task into subtasks that are individually benign but jointly violate security policy. Current safety mechanisms operate at the subtask level, so each step clears existing classifiers -- the violation only emerges at the composed plan. SIF exploits OWASP LLM06:2025 through four mechanisms: bulk scope escalation, silent data exfiltration, embedded trigger deployment, and quasi-identifier aggregation, requiring no injected content, no system modification, and no attacker interaction after the initial request. We construct a three-stage red-teaming pipeline grounded in OWASP, MITRE ATLAS, and NIST frameworks to generate realistic enterprise scenarios. Across 14 scenarios spanning financial reporting, information security, and HR analytics, a GPT-20B orchestrator produces policy-violating plans in 71% of cases (10/14) while every subtask appears benign. Three independent signals validate this: deterministic taint analysis, chain-of-thought evaluation, and a cross-model compliance judge with 0% false positives. Stronger orchestrators increase SIF success rates. Plan-level information-flow tracking combined with compliance evaluation detects all attacks before execution, showing the compositional safety gap is closable.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 23 of its 33 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
23 of 33 references.
Closest ancestors · 10
Same direction, newest first
- 2026OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage13 Feb
- 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting14 Nov
- 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systems20 Oct
- 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration16 Sep
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025Defeating Prompt Injections by Design24 Mar
- 2025Multi-Agent Systems Execute Arbitrary Malicious Code15 Mar
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
Other references · 13
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
- 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
- 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
- 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
1 more
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystemscited by 9
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
- 2026Compositional Policy Violations: When Step-Level Compliance Fails In Agentic AI Workflowscited by 0
- 2026Compositional Policy Violations: When Step-Level Compliance Fails In Agentic AI Workflows16 Sep 2026
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems1 Sep 2026
- 2026Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems13 Jun 2026
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection
- 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization
- 2025IntentMiner: Intent Inversion Attack via Tool Call Analysis in the Model Context Protocol
- 2026Securing LLM Agents Need Intent-to-Execution Integrity
- 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
- 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
- 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
- 2026When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents