# Mathematical foundations of moral preferences

arXiv: https://arxiv.org/abs/2101.08193 · HTML: https://mlsanity.com/paper/2101.08193/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2021.

## Built on by (most cited first, 16 of 16)

- [Language-based game theory in the age of artificial intelligence](https://mlsanity.com/paper/2403.08944/) (2024)
- [Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions](https://mlsanity.com/paper/2503.00940/) (2025)
- [Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifesto](https://mlsanity.com/paper/2312.01818/) (2023)
- [On the Pros and Cons of Active Learning for Moral Preference Elicitation](https://mlsanity.com/paper/2407.18889/) (2024)
- [The Physics of Preference: Unravelling Imprecision of Human Preferences through Magnetisation Dynamics](https://mlsanity.com/paper/2310.00267/) (2023)
- [From Outcome-Based to Language-Based Preferences](https://mlsanity.com/paper/2206.07300/) (2022)
- [Evolutionary mechanisms that promote cooperation may not promote social welfare](https://mlsanity.com/paper/2408.05373/) (2024)
- [Cooperation Through Indirect Reciprocity in Child-Robot Interactions](https://mlsanity.com/paper/2512.20621/) (2025)
- [Dynamics of Moral Behavior in Heterogeneous Populations of Learning Agents](https://mlsanity.com/paper/2403.04202/) (2024)
- [Emotions as abstract evaluation criteria in biological and artificial intelligences](https://mlsanity.com/paper/2111.15275/) (2021)
- [Prismal view of ethics](https://mlsanity.com/paper/2205.13370/) (2022)
- [Discrete Versus Continuous Algorithms in Dynamics of Affective Decision Making](https://mlsanity.com/paper/2309.00357/) (2023)
- [Human behaviour through a LENS: How Linguistic content triggers Emotions and Norms and determines Strategy choices](https://mlsanity.com/paper/2403.15293/) (2024)
- [Cooperative Evolutionary Pressure and Diminishing Returns Might Explain the Fermi Paradox: On What Super-AIs Are Like](https://mlsanity.com/paper/2404.03685/) (2024)
- [TUC-PPO: Team Utility-Constrained Proximal Policy Optimization for Spatial Public Goods Games](https://mlsanity.com/paper/2507.02675/) (2025)
- [The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem](https://mlsanity.com/paper/2604.14990/) (2026)

## Abstract (arXiv)

One-shot anonymous unselfishness in economic games is commonly explained by social preferences, which assume that people care about the monetary payoffs of others. However, during the last ten years, research has shown that different types of unselfish behaviour, including cooperation, altruism, truth-telling, altruistic punishment, and trustworthiness are in fact better explained by preferences for following one's own personal norms - internal standards about what is right or wrong in a given situation. Beyond better organising various forms of unselfish behaviour, this moral preference hypothesis has recently also been used to increase charitable donations, simply by means of interventions that make the morality of an action salient. Here we review experimental and theoretical work dedicated to this rapidly growing field of research, and in doing so we outline mathematical foundations for moral preferences that can be used in future models to better understand selfless human actions and to adjust policies accordingly. These foundations can also be used by artificial intelligence to better navigate the complex landscape of human morality.