# Superintelligence cannot be contained: Lessons from Computability Theory

arXiv: https://arxiv.org/abs/1607.00913 · HTML: https://mlsanity.com/paper/1607.00913/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2016.

## Built on by (most cited first, 13 of 13)

- [Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)](https://mlsanity.com/paper/2112.15422/) (2021)
- [Exploring and steering the moral compass of Large Language Models](https://mlsanity.com/paper/2405.17345/) (2024)
- [Impossibility Results in AI: A Survey](https://mlsanity.com/paper/2109.00484/) (2021)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)
- [Principles for Responsible AI Consciousness Research](https://mlsanity.com/paper/2501.07290/) (2025)
- [Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligence](https://mlsanity.com/paper/2407.03652/) (2024)
- [AI Risk Skepticism](https://mlsanity.com/paper/2105.02704/) (2021)
- [On the Undecidability of Artificial Intelligence Alignment: Machines that Halt](https://mlsanity.com/paper/2408.08995/) (2024)
- [Safety from Honesty in a Disinterested AI Predictor](https://mlsanity.com/paper/2606.29657/) (2026)
- [On Formally Undecidable Traits of Intelligent Machines](https://mlsanity.com/paper/2402.09500/) (2024)
- [The Einstein Test: Towards a Practical Test of a Machine's Ability to Exhibit Superintelligence](https://mlsanity.com/paper/2501.06948/) (2025)
- [Limitations on Accurate, Trusted, Human-level Reasoning](https://mlsanity.com/paper/2509.21654/) (2025)
- [Take Goodhart Seriously: Principled Limit on General-Purpose AI Optimization](https://mlsanity.com/paper/2510.02840/) (2025)

## Abstract (arXiv)

Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible.