mlsanityData to 14 Oct 2026

Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems

arXiv 2604.22879 ↗First posted 24 Apr 2026cs.MA cs.AI cs.CRcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

We identify and formalize a novel security risk: Context-Fragmented Violations (CFVs) - a class of policy breaches where individual agent actions appear locally safe and reasonable, yet collectively violate organizational policies because critical policy facts are siloed in different departments private contexts. Existing prompt-based alignment mechanisms and monolithic interceptors are poorly matched to violations that span contextual islands. We propose Distributed Sentinel, a distributed zero-trust enforcement architecture that introduces the Semantic Taint Token (STT) Protocol. Through lightweight sidecar proxies, our system propagates security state across organizational boundaries without exposing raw cross-domain data, enabling Counterfactual Graph Simulation for cross-domain policy verification. We construct PhantomEcosystem, a comprehensive benchmark comprising 9 categories of realistic cross-agent violation scenarios with adversarially balanced safe controls. On this benchmark, Distributed Sentinel achieves F1 = 0.95 with 106ms end-to-end latency (16ms verification + 90ms entity extraction on A100), compared to 0.85 F1 for prompt-based filtering and 0.65 for rule-based DLP. To empirically validate the need for external enforcement, we evaluate eight frontier LLMs in execution-oriented multi-agent workflows with per-agent domain world models. All models exhibit substantial violation rates (14-98%), with cross-domain data flows showing systematically higher violation rates than same-domain flows. These results indicate that self-avoidance is unreliable and that multi-agent security benefits from a centralized enforcement layer operating above individual agents.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 34 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

22 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  12. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
9 more
  1. 2022Red Teaming Language Models with Language Modelscited by 1,026
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  3. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  5. 2023Augmented Language Models: a Surveycited by 403
  6. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  7. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  8. 2024AI Agents That Mattercited by 219
  9. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 135

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Policy-Invisible Violations in LLM-Based Agents
  2. 2026Alignment Contracts for Agentic Security Systems
  3. 2026MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agents
  4. 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
  5. 2026Hybrid Inspection and Task-Based Access Control in Zero-Trust Agentic AI
  6. 2026OpenAgentFlow: Enabling System-Wide Safety Boundaries for Heterogeneous AI Agent Fleets
  7. 2025Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systems
  8. 2026Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming