# Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions

arXiv: https://arxiv.org/abs/2503.00940 · HTML: https://mlsanity.com/paper/2503.00940/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 155 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 30 of 32)

- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Power to the People? Opportunities and Challenges for Participatory AI](https://mlsanity.com/paper/2209.07572/) (2022)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Falling Rule Lists](https://mlsanity.com/paper/1411.5899/) (2014)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [What Would Jiminy Cricket Do? Towards Agents That Behave Morally](https://mlsanity.com/paper/2110.13136/) (2021)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [The social dilemma of autonomous vehicles](https://mlsanity.com/paper/1510.03346/) (2015)
- [Adapting a Kidney Exchange Algorithm to Align with Human Values](https://mlsanity.com/paper/2005.09755/) (2020)
- [What are human values, and how do we align AI to them?](https://mlsanity.com/paper/2404.10636/) (2024)
- [Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making](https://mlsanity.com/paper/2102.00625/) (2021)
- [Moral Machine or Tyranny of the Majority?](https://mlsanity.com/paper/2305.17319/) (2023)
- [Mathematical foundations of moral preferences](https://mlsanity.com/paper/2101.08193/) (2021)
- [A Computational Model of Commonsense Moral Decision Making](https://mlsanity.com/paper/1801.04346/) (2018)
- [The Conflict Between Explainable and Accountable Decision-Making Algorithms](https://mlsanity.com/paper/2205.05306/) (2022)
- [On the Strategyproofness of the Geometric Median](https://mlsanity.com/paper/2106.02394/) (2021)
- [Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities](https://mlsanity.com/paper/1910.13122/) (2019)
- [Indecision Modeling](https://mlsanity.com/paper/2012.08485/) (2020)
- [Deploying a Robust Active Preference Elicitation Algorithm on MTurk: Experiment Design, Interface, and Evaluation for COVID-19 Patient Prioritization](https://mlsanity.com/paper/2306.04061/) (2023)
- [On the Pros and Cons of Active Learning for Moral Preference Elicitation](https://mlsanity.com/paper/2407.18889/) (2024)
- [On The Stability of Moral Preferences: A Problem with Computational Elicitation Methods](https://mlsanity.com/paper/2408.02862/) (2024)
- [Minimum Levels of Interpretability for Artificial Moral Agents](https://mlsanity.com/paper/2307.00660/) (2023)
- [Diverse Representation via Computational Participatory Elections -- Lessons from a Case Study](https://mlsanity.com/paper/2205.15394/) (2022)
- [Developing moral AI to support antimicrobial decision making](https://mlsanity.com/paper/2208.06327/) (2022)

## Built on by (most cited first, 9 of 9)

- [Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback](https://mlsanity.com/paper/2511.10032/) (2025)
- [Position: Machine Learning for Heart Transplant Allocation Policy Optimization Should Account for Incentives](https://mlsanity.com/paper/2602.04990/) (2026)
- [Towards Cognitively-Faithful Decision-Making Models to Improve AI Alignment](https://mlsanity.com/paper/2509.04445/) (2025)
- [Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values](https://mlsanity.com/paper/2506.00079/) (2025)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Evaluating Large Language Models for Fair and Reliable Organ Allocation](https://mlsanity.com/paper/2504.03716/) (2025)
- [Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values](https://mlsanity.com/paper/2608.28620/) (2026)
- [Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences](https://mlsanity.com/paper/2605.16615/) (2026)
- [Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning](https://mlsanity.com/paper/2608.12372/) (2026)

## Abstract (arXiv)

A growing body of work in Ethical AI attempts to capture human moral judgments through simple computational models. The key question we address in this work is whether such simple AI models capture {the critical} nuances of moral decision-making by focusing on the use case of kidney allocation. We conducted twenty interviews where participants explained their rationale for their judgments about who should receive a kidney. We observe participants: (a) value patients' morally-relevant attributes to different degrees; (b) use diverse decision-making processes, citing heuristics to reduce decision complexity; (c) can change their opinions; (d) sometimes lack confidence in their decisions (e.g., due to incomplete information); and (e) express enthusiasm and concern regarding AI assisting humans in kidney allocation decisions. Based on these findings, we discuss challenges of computationally modeling moral judgments {as a stand-in for human input}, highlight drawbacks of current approaches, and suggest future directions to address these issues.