# Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values

arXiv: https://arxiv.org/abs/2506.00079 · HTML: https://mlsanity.com/paper/2506.00079/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 36 of its 80 references that are in the map.
- First posted on arXiv 30 May 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 30 of 36)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [Are Large Language Models Consistent over Value-laden Questions?](https://mlsanity.com/paper/2407.02996/) (2024)
- [Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models](https://mlsanity.com/paper/2407.16221/) (2024)
- [Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions](https://mlsanity.com/paper/2504.15236/) (2025)
- [Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?](https://mlsanity.com/paper/2501.15463/) (2025)
- [MoralBench: Moral Evaluation of LLMs](https://mlsanity.com/paper/2406.04428/) (2024)
- [Forcing Diffuse Distributions out of Language Models](https://mlsanity.com/paper/2404.10859/) (2024)
- [A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine](https://mlsanity.com/paper/2405.08603/) (2024)
- [Exploring the psychology of LLMs' Moral and Legal Reasoning](https://mlsanity.com/paper/2308.01264/) (2023)
- [Generative Social Choice](https://mlsanity.com/paper/2309.01291/) (2023)
- [Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context](https://mlsanity.com/paper/2406.05972/) (2024)
- [Moral Alignment for LLM Agents](https://mlsanity.com/paper/2410.01639/) (2024)
- [Adapting a Kidney Exchange Algorithm to Align with Human Values](https://mlsanity.com/paper/2005.09755/) (2020)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs](https://mlsanity.com/paper/2409.09586/) (2024)
- [Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy](https://mlsanity.com/paper/2503.09639/) (2025)
- [Personality-Driven Decision-Making in LLM-Based Autonomous Agents](https://mlsanity.com/paper/2504.00727/) (2025)
- [The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making](https://mlsanity.com/paper/2410.07304/) (2024)
- [Generative Social Choice: The Next Generation](https://mlsanity.com/paper/2505.22939/) (2025)
- [Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions](https://mlsanity.com/paper/2503.00940/) (2025)
- [Indecision Modeling](https://mlsanity.com/paper/2012.08485/) (2020)
- [On The Stability of Moral Preferences: A Problem with Computational Elicitation Methods](https://mlsanity.com/paper/2408.02862/) (2024)

## Built on by (most cited first, 2 of 2)

- [Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback](https://mlsanity.com/paper/2511.10032/) (2025)
- [Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values](https://mlsanity.com/paper/2608.28620/) (2026)

## Abstract (arXiv)

The rapid integration of Large Language Models (LLMs) in high-stakes decision-making -- such as allocating scarce resources like donor organs -- raises critical questions about their alignment with human moral values. We systematically evaluate the behavior of several prominent LLMs against human preferences in kidney allocation scenarios and show that LLMs: i) exhibit stark deviations from human values in prioritizing various attributes, and ii) in contrast to humans, LLMs rarely express indecision, opting for deterministic decisions even when alternative indecision mechanisms (e.g., coin flipping) are provided. Nonetheless, we show that low-rank supervised fine-tuning with few samples is often effective in improving both decision consistency and calibrating indecision modeling. These findings illustrate the necessity of explicit alignment strategies for LLMs in moral/ethical domains.