# Low Impact Artificial Intelligences

arXiv: https://arxiv.org/abs/1705.10720 · HTML: https://mlsanity.com/paper/1705.10720/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2017.

## Built on by (most cited first, 11 of 11)

- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI](https://mlsanity.com/paper/2306.06924/) (2023)
- [Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory](https://mlsanity.com/paper/2602.18297/) (2026)
- [Designing Fiduciary Artificial Intelligence](https://mlsanity.com/paper/2308.02435/) (2023)
- [Combining Reward Information from Multiple Sources](https://mlsanity.com/paper/2103.12142/) (2021)
- [AI Agents Should be Regulated Based on the Extent of Their Autonomous Operations](https://mlsanity.com/paper/2503.04750/) (2025)
- [Challenges for Using Impact Regularizers to Avoid Negative Side Effects](https://mlsanity.com/paper/2101.12509/) (2021)
- [Improving performance in multi-objective decision-making in Bottles environments with soft maximin approaches](https://mlsanity.com/paper/2208.04273/) (2022)
- [Low impact agency: review and discussion](https://mlsanity.com/paper/2303.03139/) (2023)
- [Shutdown Safety Valves for Advanced AI](https://mlsanity.com/paper/2603.07315/) (2026)
- [Calibrating Conservatism for Scalable Oversight](https://mlsanity.com/paper/2605.28807/) (2026)

## Abstract (arXiv)

There are many goals for an AI that could become dangerous if the AI becomes superintelligent or otherwise powerful. Much work on the AI control problem has been focused on constructing AI goals that are safe even for such AIs. This paper looks at an alternative approach: defining a general concept of `low impact'. The aim is to ensure that a powerful AI which implements low impact will not modify the world extensively, even if it is given a simple or dangerous goal. The paper proposes various ways of defining and grounding low impact, and discusses methods for ensuring that the AI can still be allowed to have a (desired) impact despite the restriction. The end of the paper addresses known issues with this approach and avenues for future research.