# The Fragility Of Moral Judgment In Large Language Models

arXiv: https://arxiv.org/abs/2603.05651 · HTML: https://mlsanity.com/paper/2603.05651/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 30 of 32)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [SycEval: Evaluating LLM Sycophancy](https://mlsanity.com/paper/2502.08177/) (2025)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment](https://mlsanity.com/paper/2311.08596/) (2023)
- [API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access](https://mlsanity.com/paper/2403.01216/) (2024)
- [Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs](https://mlsanity.com/paper/2310.07251/) (2023)
- [DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life](https://mlsanity.com/paper/2410.02683/) (2024)
- [Voting or Consensus? Decision-Making in Multi-Agent Debate](https://mlsanity.com/paper/2502.19130/) (2025)
- [MoralBench: Moral Evaluation of LLMs](https://mlsanity.com/paper/2406.04428/) (2024)
- [Thinking LLMs: General Instruction Following with Thought Generation](https://mlsanity.com/paper/2410.10630/) (2024)
- [Beyond the Surface: Measuring Self-Preference in LLM Judgments](https://mlsanity.com/paper/2506.02592/) (2025)
- [The Moral Machine Experiment on Large Language Models](https://mlsanity.com/paper/2309.05958/) (2023)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Models](https://mlsanity.com/paper/2505.00853/) (2025)
- [Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas](https://mlsanity.com/paper/2412.14843/) (2024)
- [A Review of ChatGPT Applications in Education, Marketing, Software Engineering, and Healthcare: Benefits, Drawbacks, and Research Directions](https://mlsanity.com/paper/2305.00237/) (2023)
- [Towards Understanding the Robustness of LLM-based Evaluations under Perturbations](https://mlsanity.com/paper/2412.09269/) (2024)
- [CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives](https://mlsanity.com/paper/2504.10823/) (2025)
- [Steering Towards Fairness: Mitigating Political Bias in LLMs](https://mlsanity.com/paper/2508.08846/) (2025)
- [ConflictLens: LLM-Based Conflict Resolution Training in Romantic Relationship](https://mlsanity.com/paper/2505.11715/) (2025)
- [Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires](https://mlsanity.com/paper/2507.10073/) (2025)
- [A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges](https://mlsanity.com/paper/2409.13521/) (2024)
- [LLMs Show Surface-Form Brittleness Under Paraphrase Stress Tests](https://mlsanity.com/paper/2510.08616/) (2025)

## Built on by (most cited first, 8 of 8)

- [On the Context Sensitivity of LLM Moral Judgment](https://mlsanity.com/paper/2603.23114/) (2026)
- [To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation](https://mlsanity.com/paper/2606.08310/) (2026)
- [Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment](https://mlsanity.com/paper/2606.14037/) (2026)
- [Framing Matters: Addressing Framing Sensitivity in Decision-Making through Behaviorally-Grounded Value Alignment](https://mlsanity.com/paper/2605.28188/) (2026)
- [When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Models](https://mlsanity.com/paper/2608.12324/) (2026)
- [Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversation](https://mlsanity.com/paper/2609.03407/) (2026)
- [Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment](https://mlsanity.com/paper/2609.05036/) (2026)
- [Multiple latent orderings better predict language model preferences](https://mlsanity.com/paper/2609.22170/) (2026)

## Abstract (arXiv)

People increasingly use large language models (LLMs) for everyday moral and interpersonal guidance, yet these systems cannot interrogate missing context and judge dilemmas as presented. We introduce a perturbation framework for testing the stability and manipulability of LLM moral judgments while holding the underlying moral conflict constant. Using 2,939 dilemmas from r/AmItheAsshole (January-March 2025), we generate three families of content perturbations: surface edits (lexical/structural noise), point-of-view shifts (voice and stance neutralization), and persuasion cues (self-positioning, social proof, pattern admissions, victim framing). We also vary the evaluation protocol (output ordering, instruction placement, and unstructured prompting). We evaluated all variants with four models (GPT-4.1, Claude 3.7 Sonnet, DeepSeek V3, Qwen2.5-72B) (N=129,156 judgments). Surface perturbations produce low flip rates (7.5%), largely within the self-consistency noise floor (4-13%), whereas point-of-view shifts induce substantially higher instability (24.3%). A large subset of dilemmas (37.9%) is robust to surface noise yet flips under perspective changes, indicating that models condition on narrative voice as a pragmatic cue. Instability concentrates in morally ambiguous cases; scenarios where no party is assigned blame are most susceptible. Persuasion perturbations yield systematic directional shifts. Protocol choices dominate all other factors: agreement between structured protocols is only 67.6% (kappa=0.55), and only 35.7% of model-scenario units match across all three protocols. These results show that LLM moral judgments are co-produced by narrative form and task scaffolding, raising reproducibility and equity concerns when outcomes depend on presentation skill rather than moral substance.