# Morality, Machines and the Interpretation Problem: A Value-based, Wittgensteinian Approach to Building Moral Agents

arXiv: https://arxiv.org/abs/2103.02728 · HTML: https://mlsanity.com/paper/2103.02728/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 35 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 2 of 2)

- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Avoiding Wireheading with Value Reinforcement Learning](https://mlsanity.com/paper/1605.03143/) (2016)

## Built on by (most cited first, 6 of 6)

- [Understanding and Avoiding AI Failures: A Practical Guide](https://mlsanity.com/paper/2104.12582/) (2021)
- [Minimum Levels of Interpretability for Artificial Moral Agents](https://mlsanity.com/paper/2307.00660/) (2023)
- [Breaking Bad News in the Era of Artificial Intelligence and Algorithmic Medicine: An Exploration of Disclosure and its Ethical Justification using the Hedonic Calculus](https://mlsanity.com/paper/2207.01431/) (2022)
- [Establishing Meta-Decision-Making for AI: An Ontology of Relevance, Representation and Reasoning](https://mlsanity.com/paper/2210.00608/) (2022)
- [If our aim is to build morality into an artificial agent, how might we begin to go about doing so?](https://mlsanity.com/paper/2310.08295/) (2023)
- [Can a Machine be Conscious? Towards Universal Criteria for Machine Consciousness](https://mlsanity.com/paper/2404.15369/) (2024)

## Abstract (arXiv)

We present what we call the Interpretation Problem, whereby any rule in symbolic form is open to infinite interpretation in ways that we might disapprove of and argue that any attempt to build morality into machines is subject to it. We show how the Interpretation Problem in Artificial Intelligence is an illustration of Wittgenstein's general claim that no rule can contain the criteria for its own application, and that the risks created by this problem escalate in proportion to the degree to which to machine is causally connected to the world, in what we call the Law of Interpretative Exposure. Using game theory, we attempt to define the structure of normative spaces and argue that any rule-following within a normative space is guided by values that are external to that space and which cannot themselves be represented as rules. In light of this, we categorise the types of mistakes an artificial moral agent could make into Mistakes of Intention and Instrumental Mistakes, and we propose ways of building morality into machines by getting them to interpret the rules we give in accordance with these external values, through explicit moral reasoning, the Show, not Tell paradigm, the adjustment of causal power and structure of the agent, and relational values, with the ultimate aim that the machine develop a virtuous character and that the impact of the Interpretation Problem is minimised.