# Death and Suicide in Universal Artificial Intelligence

arXiv: https://arxiv.org/abs/1606.00652 · HTML: https://mlsanity.com/paper/1606.00652/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2016.

## Built on by (most cited first, 9 of 9)

- [Human Control: Definitions and Algorithms](https://mlsanity.com/paper/2305.19861/) (2023)
- [Reward-Punishment Symmetric Universal Intelligence](https://mlsanity.com/paper/2110.02450/) (2021)
- [Value Under Ignorance in Universal Artificial Intelligence](https://mlsanity.com/paper/2512.17086/) (2025)
- [Quantifying stability of non-power-seeking in artificial agents](https://mlsanity.com/paper/2401.03529/) (2024)
- [Towards Shutdownable Agents via Stochastic Choice](https://mlsanity.com/paper/2407.00805/) (2024)
- [Solomonoff induction](https://mlsanity.com/paper/2603.20274/) (2026)
- [Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs](https://mlsanity.com/paper/2604.17502/) (2026)
- [A Roadmap for Robust End-to-End Alignment](https://mlsanity.com/paper/1809.01036/) (2018)
- [Shutdown Safety Valves for Advanced AI](https://mlsanity.com/paper/2603.07315/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) is a general paradigm for studying intelligent behaviour, with applications ranging from artificial intelligence to psychology and economics. AIXI is a universal solution to the RL problem; it can learn any computable environment. A technical subtlety of AIXI is that it is defined using a mixture over semimeasures that need not sum to 1, rather than over proper probability measures. In this work we argue that the shortfall of a semimeasure can naturally be interpreted as the agent's estimate of the probability of its death. We formally define death for generally intelligent agents like AIXI, and prove a number of related theorems about their behaviour. Notable discoveries include that agent behaviour can change radically under positive linear transformations of the reward signal (from suicidal to dogmatically self-preserving), and that the agent's posterior belief that it will survive increases over time.