mlsanityData to 14 Oct 2026

Moral Uncertainty and the Problem of Fanaticism

arXiv 2312.11589 ↗First posted 18 Dec 2023cs.AI cs.GT econ.THcited by 1

Not on the map. Why?

Abstract (arXiv)

While there is universal agreement that agents ought to act ethically, there is no agreement as to what constitutes ethical behaviour. To address this problem, recent philosophical approaches to `moral uncertainty' propose aggregation of multiple ethical theories to guide agent behaviour. However, one of the foundational proposals for aggregation - Maximising Expected Choiceworthiness (MEC) - has been criticised as being vulnerable to fanaticism; the problem of an ethical theory dominating agent behaviour despite low credence (confidence) in said theory. Fanaticism thus undermines the `democratic' motivation for accommodating multiple ethical perspectives. The problem of fanaticism has not yet been mathematically defined. Representing moral uncertainty as an instance of social welfare aggregation, this paper contributes to the field of moral uncertainty by 1) formalising the problem of fanaticism as a property of social welfare functionals and 2) providing non-fanatical alternatives to MEC, i.e. Highest k-trimmed Mean and Highest Median.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 3 of its 22 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

3 of 22 references.

References · 3

  1. 2020Artificial Intelligence, Values and Alignmentcited by 448
  2. 2016Cooperative Inverse Reinforcement Learningcited by 300
  3. 2020Reinforcement Learning Under Moral Uncertaintycited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.