mlsanityData to 14 Oct 2026

The Fragility Of Moral Judgment In Large Language Models

arXiv 2603.05651 ↗First posted 5 Mar 2026cs.CL cs.AI cs.HCcited by 8

Not on the map. Why?

Abstract (arXiv)

People increasingly use large language models (LLMs) for everyday moral and interpersonal guidance, yet these systems cannot interrogate missing context and judge dilemmas as presented. We introduce a perturbation framework for testing the stability and manipulability of LLM moral judgments while holding the underlying moral conflict constant. Using 2,939 dilemmas from r/AmItheAsshole (January-March 2025), we generate three families of content perturbations: surface edits (lexical/structural noise), point-of-view shifts (voice and stance neutralization), and persuasion cues (self-positioning, social proof, pattern admissions, victim framing). We also vary the evaluation protocol (output ordering, instruction placement, and unstructured prompting). We evaluated all variants with four models (GPT-4.1, Claude 3.7 Sonnet, DeepSeek V3, Qwen2.5-72B) (N=129,156 judgments). Surface perturbations produce low flip rates (7.5%), largely within the self-consistency noise floor (4-13%), whereas point-of-view shifts induce substantially higher instability (24.3%). A large subset of dilemmas (37.9%) is robust to surface noise yet flips under perspective changes, indicating that models condition on narrative voice as a pragmatic cue. Instability concentrates in morally ambiguous cases; scenarios where no party is assigned blame are most susceptible. Persuasion perturbations yield systematic directional shifts. Protocol choices dominate all other factors: agreement between structured protocols is only 67.6% (kappa=0.55), and only 35.7% of model-scenario units match across all three protocols. These results show that LLM moral judgments are co-produced by narrative form and task scaffolding, raising reproducibility and equity concerns when outcomes depend on presentation skill rather than moral substance.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 53 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

32 of 53 references.

References · 32

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  3. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  4. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  5. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  6. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  7. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 149
  8. 2025SycEval: Evaluating LLM Sycophancycited by 108
  9. 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 90
  10. 2023Moral Foundations of Large Language Modelscited by 85
  11. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
  12. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
20 more
  1. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
  2. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  3. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 59
  4. 2024MoralBench: Moral Evaluation of LLMscited by 50
  5. 2024Thinking LLMs: General Instruction Following with Thought Generationcited by 45
  6. 2025Beyond the Surface: Measuring Self-Preference in LLM Judgmentscited by 31
  7. 2023The Moral Machine Experiment on Large Language Modelscited by 25
  8. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 24
  9. 2025LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Modelscited by 21
  10. 2024Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personascited by 16
  11. 2023A Review of ChatGPT Applications in Education, Marketing, Software Engineering, and Healthcare: Benefits, Drawbacks, and Research Directionscited by 12
  12. 2024Towards Understanding the Robustness of LLM-based Evaluations under Perturbationscited by 10
  13. 2025CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectivescited by 10
  14. 2025Steering Towards Fairness: Mitigating Political Bias in LLMscited by 7
  15. 2025ConflictLens: LLM-Based Conflict Resolution Training in Romantic Relationshipcited by 4
  16. 2025Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnairescited by 4
  17. 2024A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challengescited by 3
  18. 2025LLMs Show Surface-Form Brittleness Under Paraphrase Stress Testscited by 3
  19. 2025Structured Prompts Improve Evaluation of Language Modelscited by 3
  20. 2025A methodological analysis of prompt perturbations and their effect on attack success ratescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026On the Context Sensitivity of LLM Moral Judgmentcited by 3
  2. 2026To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulationcited by 1
  3. 2026Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgmentcited by 1
  4. 2026Framing Matters: Addressing Framing Sensitivity in Decision-Making through Behaviorally-Grounded Value Alignmentcited by 0
  5. 2026When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Modelscited by 0
  6. 2026Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversationcited by 0
  7. 2026Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignmentcited by 0
  8. 2026Multiple latent orderings better predict language model preferencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Moral Persuasion in Large Language Models: Evaluating Susceptibility and Ethical Alignment
  2. 2026Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models
  3. 2025From Stability to Inconsistency: A Study of Moral Preferences in LLMs
  4. 2025"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas
  5. 2025Diagnosing Moral Reasoning Acquisition in Language Models: Pragmatics and Generalization
  6. 2023MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks
  7. 2026Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments
  8. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning