# The Accountability Horizon: An Impossibility Theorem for Governing Human-Agent Collectives

arXiv: https://arxiv.org/abs/2604.07778 · HTML: https://mlsanity.com/paper/2604.07778/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 41 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 4 of 4)

- [Foundations of Structural Causal Models with Cycles and Latent Variables](https://mlsanity.com/paper/1611.06221/) (2016)
- [Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)](https://mlsanity.com/paper/1901.00064/) (2018)
- [Runtime Governance for AI Agents: Policies on Paths](https://mlsanity.com/paper/2603.16586/) (2026)
- [Impossibility Results in AI: A Survey](https://mlsanity.com/paper/2109.00484/) (2021)

## Built on by (most cited first, 1 of 1)

- [Governing Agentic AI in FinTech](https://mlsanity.com/paper/2608.11344/) (2026)

## Abstract (arXiv)

Existing accountability frameworks for AI systems, legal, ethical, and regulatory, rest on a shared assumption: for any consequential outcome, at least one identifiable person had enough involvement and foresight to bear meaningful responsibility. This paper proves that agentic AI systems violate this assumption not as an engineering limitation but as a mathematical necessity once autonomy exceeds a computable threshold. We introduce Human-Agent Collectives, a formalisation of joint human-AI systems where agents are modelled as state-policy tuples within a shared structural causal model. Autonomy is characterised through a four-dimensional information-theoretic profile (epistemic, executive, evaluative, social); collective behaviour through interaction graphs and joint action spaces. We axiomatise legitimate accountability through four minimal properties: Attributability (responsibility requires causal contribution), Foreseeability Bound (responsibility cannot exceed predictive capacity), Non-Vacuity (at least one agent bears non-trivial responsibility), and Completeness (all responsibility must be fully allocated). Our central result, the Accountability Incompleteness Theorem, proves that for any collective whose compound autonomy exceeds the Accountability Horizon and whose interaction graph contains a human-AI feedback cycle, no framework can satisfy all four properties simultaneously. The impossibility is structural: transparency, audits, and oversight cannot resolve it without reducing autonomy. Below the threshold, legitimate frameworks exist, establishing a sharp phase transition. Experiments on 3,000 synthetic collectives confirm all predictions with zero violations. This is the first impossibility result in AI governance, establishing a formal boundary below which current paradigms remain valid and above which distributed accountability mechanisms become necessary.