# Arguments about Highly Reliable Agent Designs as a Useful Path to Artificial Intelligence Safety

arXiv: https://arxiv.org/abs/2201.02950 · HTML: https://mlsanity.com/paper/2201.02950/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 43 references that are in the map.
- First posted on arXiv 9 Jan 2022.

## Builds on (most cited first, 6 of 6)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Embedded Agency](https://mlsanity.com/paper/1902.09469/) (2019)
- [Logical Induction](https://mlsanity.com/paper/1609.03543/) (2016)
- [Functional Decision Theory: A New Theory of Instrumental Rationality](https://mlsanity.com/paper/1710.05060/) (2017)
- [AI Risk Skepticism](https://mlsanity.com/paper/2105.02704/) (2021)

## Abstract (arXiv)

Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of different approaches have made different and debated claims about the importance or usefulness of their work in the near term, and for future systems. Highly Reliable Agent Designs (HRAD) is one of the most controversial and ambitious approaches, championed by the Machine Intelligence Research Institute, among others, and various arguments have been made about whether and how it reduces risks from future AI systems. In order to reduce confusion in the debate about AI safety, here we build on a previous discussion by Rice which collects and presents four central arguments which are used to justify HRAD as a path towards safety of AI systems. We have titled the arguments (1) incidental utility,(2) deconfusion, (3) precise specification, and (4) prediction. Each of these makes different, partly conflicting claims about how future AI systems can be risky. We have explained the assumptions and claims based on a review of published and informal literature, along with consultation with experts who have stated positions on the topic. Finally, we have briefly outlined arguments against each approach and against the agenda overall.