Some[Body] Must Receive That Pain for Agent Accountability
Not on the map. Why?
Abstract (arXiv)
AI agents increasingly act consequentially in the real world. This creates a problem we call consequence reception: harm occurs, the producing system is identified, yet no continuing agent receives consequences in a way that changes future behavior. Pain, understood mechanistically as a corrective feedback signal, is foundational to canonical theories of punishment -- deterrence, rehabilitation, retribution, and incapacitation all assume a continuing locus that registers the signal and updates behavior. That, in turn, requires a body for the signal to land on: a boundary whose integrity it protects, a locus where it accumulates, consolidation that converts episodic signal into durable update, and a substrate that responds by altering future action. Current LLM agents -- software-defined composites of weights, prompts, tools, memory, and credentials, freely swapped, copied, reset, and reassembled -- satisfy none of these conditions. The two prevailing legal responses therefore fail to achieve consequence reception. The thin-identity agent-principal dyad has a body but no consequence--agency coupling: the human bears pain for behaviors beyond their control -- Elish's moral crumple zone. The thick-identity Arbel et al.'s Algorithmic Corporation creates legally legible entities but does not guarantee that any AI decision architecture receives pain as a behavioral signal. Achieving consequence-agency coupling is therefore a sociotechnical infrastructural problem, not only a legal one. Until such architectures exist, high-stakes AI deployment should remain tethered to accountable human principals with meaningful control, proportional liability, and authority to constrain or terminate the agent. If some body does not receive the pain by design, some body will receive it by default.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 70 references that are in the map. First posted on arXiv 16 May 2026.
Builds on
26 of 70 references.
References · 26
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
- 2016Stealing Machine Learning Models via Prediction APIscited by 602
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
- 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
14 more
- 2024Frontier Models are Capable of In-context Schemingcited by 252
- 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 136
- 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
- 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
- 2026Emotion Concepts and their Function in a Large Language Modelcited by 48
- 2023Understanding accountability in algorithmic supply chainscited by 39
- 2024Taking AI Welfare Seriouslycited by 28
- 2024Frontier AI systems have surpassed the self-replicating red linecited by 16
- 2025Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMscited by 16
- 2023Mortal Computation: A Foundation for Biomimetic Intelligencecited by 15
- 2026Sovereign Agents: Towards Infrastructural Sovereignty and Diffused Accountability in Decentralized AIcited by 6
- 2026How to Count AIs: Individuation and Liability for AI Agentscited by 5
- 2025AI Wellbeingcited by 4
- 2014Do Artificial Reinforcement-Learning Agents Matter Morally?cited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026One Gate Is Not Enough: Composing Stateful Pre-Action Controls for Agentic AI
- 2025From surveillance to signalling: escalation channels as environmental controls for agentic AI
- 2026Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing Human
- 2026Emotional Cost Functions for AI Safety: Teaching Agents to Feel the Weight of Irreversible Consequences
- 2026When Should an AI Act? A Human-Centered Model of Scene, Context, and Behavior for Agentic AI Design
- 2026The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents
- 2026Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors
- 2025Moral Responsibility or Obedience: What Do We Want from AI?