# Accounting for Context: Shaping Moral Credences for Value Alignment

arXiv: https://arxiv.org/abs/2606.06972 · HTML: https://mlsanity.com/paper/2606.06972/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 29 references that are in the map.
- First posted on arXiv 5 Jun 2026.

## Builds on (most cited first, 5 of 5)

- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Representative Social Choice: From Learning Theory to AI Alignment](https://mlsanity.com/paper/2410.23953/) (2024)
- [On the Computational Complexity of Ethics: Moral Tractability for Minds and Machines](https://mlsanity.com/paper/2302.04218/) (2023)
- [Moral Uncertainty and the Problem of Fanaticism](https://mlsanity.com/paper/2312.11589/) (2023)

## Abstract (arXiv)

Ensuring that agent behaviours are aligned with human moral values inevitably raises the problem of how to account for the plurality of moral perspectives that societies -- and even individuals -- typically adopt. Work on moral uncertainty proposes mechanisms to fairly and democratically aggregate evaluations of actions across different moral theories. However, this paper argues that one needs to account for contextual factors when aggregating moral evaluations. For example, consequentialist perspectives assume an ability to accurately determine how an agent's actions change the world; an assumption that often does not hold in real world settings. We, therefore, formalise agent decision making under moral uncertainty, while also accounting for these kinds of contextual factors. We thereby show that a seemingly commonsensical property -- the weak Pareto principle -- is violated. We argue that this apparent problem is, in fact, a variation of Simpson's paradox, and hence reveals the limitations of aggregation mechanisms that ignore the impact of contextual factors.