How to Count AIs: Individuation and Liability for AI Agents
Not on the map. Why?
Abstract (arXiv)
Very soon, millions of AI agents will proliferate across the economy, autonomously taking billions of actions. Inevitably, things will go wrong. Humans will be defrauded, injured, even killed. Law will somehow have to govern the coming wave. But when an AI causes harm, the first question to answer, before anyone can be held accountable is: Which AI Did It? Identifying AIs is unusually difficult. AIs lack bodies. They can copy, split, merge, swarm, and vanish at will. Even today, a "single" AI agent is often an ensemble of instances based on multiple models. The complexity will only multiply as AI capabilities improve. This Article is the first to comprehensively diagnose the legal problem of identifying AIs. Two kinds of identity are required: "thin" and "thick." Thin identification ties every AI action to some human principal, essential for holding accountable the humans who make and use AI agents. Thick identification distinguishes between AI agents, qua agents -- sorting millions of AI entities into discrete, persistent units with stable, coherent goals, essential where principal-agent problems prevent humans from perfectly controlling AIs. This Article also presents a solution: the "Algorithmic Corporation" or "A-corp" -- a legal-fictional entity that can hold property, make contracts, and litigate in its own name. Owned by humans but run by AIs, A-corps solve the thin identity problem by tying AI actions to a human owner, and the thick identity problem via emergent self-organization. A-corps own the resources -- including compute -- that AIs need to accomplish their goals, giving AI managers strong incentives to share control only with goal-aligned AIs. In equilibrium, incentive and selection mechanisms force A-corps to self-organize into persistent, legally legible entities with coherent goals that respond rationally to legal incentives, like liability.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 12 of its 103 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
12 of 103 references.
References · 12
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
- 2024Alignment faking in large language modelscited by 360
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
- 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
- 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
- 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
- 2025Gradual Disempowerment: Systemic Existential Risks from Incremental AI Developmentcited by 46
- 2024IDs for AI Systemscited by 19
- 2025AI-native Memory 2.0: Second Mecited by 14
- 2025Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Frameworkcited by 5
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanismscited by 4
- 2026Incoherent Values? Probing LLM Preferences Through Parametric Variationcited by 2
- 2026SoulAuth: An Actor-native Identity Architecture and Rust Reference Implementation for Humans and Long-lived AI Actorscited by 1
- 2026Some[Body] Must Receive That Pain for Agent Accountabilitycited by 0
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocolcited by 0
- 2026SoulAuth: An Actor-native Identity Architecture and Rust Reference Implementation for Humans and Long-lived AI Actors10 Sep 2026
- 2026Incoherent Values? Probing LLM Preferences Through Parametric Variation19 Jun 2026
- 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms28 May 2026
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol23 May 2026
- 2026Some[Body] Must Receive That Pain for Agent Accountability16 May 2026