# Safety is Non-Compositional: A Formal Framework for Capability-Based AI Systems

arXiv: https://arxiv.org/abs/2603.15973 · HTML: https://mlsanity.com/paper/2603.15973/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 49 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 5 of 5)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [TaskBench: Benchmarking Large Language Models for Task Automation](https://mlsanity.com/paper/2311.18760/) (2023)

## Built on by (most cited first, 4 of 4)

- [When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.00448/) (2026)
- [Capability Safety as Datalog: A Foundational Equivalence](https://mlsanity.com/paper/2603.26725/) (2026)
- [Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelines](https://mlsanity.com/paper/2609.20211/) (2026)
- [Safety as Computation: Certified Answer Reuse via Capability Closure in Task-Oriented Dialogue](https://mlsanity.com/paper/2603.21448/) (2026)

## Abstract (arXiv)

This paper contains the first formal proof that safety is non-compositional in the presence of conjunctive capability dependencies: two agents each individually inca- pable of reaching any forbidden capability can, when combined, collectively reach a forbidden goal through an emergent conjunctive dependency.