mlsanityData to 7 Oct 2026

Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines

arXiv 2604.08608 ↗First posted 8 Apr 2026cs.CR cs.AI cs.LGcited by 4

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

We introduce Semantic Intent Fragmentation (SIF), an attack class against LLM orchestration systems where a single, legitimately phrased request causes an orchestrator to decompose a task into subtasks that are individually benign but jointly violate security policy. Current safety mechanisms operate at the subtask level, so each step clears existing classifiers -- the violation only emerges at the composed plan. SIF exploits OWASP LLM06:2025 through four mechanisms: bulk scope escalation, silent data exfiltration, embedded trigger deployment, and quasi-identifier aggregation, requiring no injected content, no system modification, and no attacker interaction after the initial request. We construct a three-stage red-teaming pipeline grounded in OWASP, MITRE ATLAS, and NIST frameworks to generate realistic enterprise scenarios. Across 14 scenarios spanning financial reporting, information security, and HR analytics, a GPT-20B orchestrator produces policy-violating plans in 71% of cases (10/14) while every subtask appears benign. Three independent signals validate this: deterministic taint analysis, chain-of-thought evaluation, and a cross-model compliance judge with 0% false positives. Stronger orchestrators increase SIF success rates. Plan-level information-flow tracking combined with compliance evaluation detects all attacks before execution, showing the compositional safety gap is closable.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 23 of its 33 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

23 of 33 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage13 Feb
  2. 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting14 Nov
  3. 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systems20 Oct
  4. 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration16 Sep
  5. 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
  6. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  7. 2025Defeating Prompt Injections by Design24 Mar
  8. 2025Multi-Agent Systems Execute Arbitrary Malicious Code15 Mar
  9. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  10. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul

Other references · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  7. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  8. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  9. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  10. 2025Securing AI Agents with Information-Flow Controlcited by 108
  11. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
  12. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
1 more
  1. 2025Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solutioncited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection
  2. 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization
  3. 2025IntentMiner: Intent Inversion Attack via Tool Call Analysis in the Model Context Protocol
  4. 2026Securing LLM Agents Need Intent-to-Execution Integrity
  5. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
  6. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
  7. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
  8. 2026When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents