# Diagnosing Moral Reasoning Acquisition in Language Models: Pragmatics and Generalization

arXiv: https://arxiv.org/abs/2502.16600 · HTML: https://mlsanity.com/paper/2502.16600/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 15 of 15)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Beyond Preferences in AI Alignment](https://mlsanity.com/paper/2408.16984/) (2024)
- [Moral Alignment for LLM Agents](https://mlsanity.com/paper/2410.01639/) (2024)
- [Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophy](https://mlsanity.com/paper/2205.12771/) (2022)
- [Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral](https://mlsanity.com/paper/2502.14083/) (2025)
- [ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models](https://mlsanity.com/paper/2310.00378/) (2023)
- [SaGE: Evaluating Moral Consistency in Large Language Models](https://mlsanity.com/paper/2402.13709/) (2024)

## Built on by (most cited first, 6 of 6)

- [Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment](https://mlsanity.com/paper/2603.11388/) (2026)
- [Discourse Heuristics For Paradoxically Moral Self-Correction](https://mlsanity.com/paper/2507.00985/) (2025)
- [Diagnosing the Performance Trade-off in Moral Alignment: A Case Study on Gender Stereotypes](https://mlsanity.com/paper/2509.21456/) (2025)
- [From Training to Generalization: Improving Moral Reasoning Through Pragmatic Inference](https://mlsanity.com/paper/2509.24102/) (2025)
- [Learning to Diagnose and Correct Moral Errors: Beyond Shallow Heuristics in Moral Alignment](https://mlsanity.com/paper/2601.03079/) (2026)
- [Pragmatic Attack Surface: Vulnerabilities of Implicit Context in Large Language Models](https://mlsanity.com/paper/2608.09551/) (2026)

## Abstract (arXiv)

Ensuring that Large Language Models (LLMs) return just responses which adhere to societal values is crucial for their broader application. Prior research has shown that LLMs often fail to perform satisfactorily on tasks requiring moral cognizance, such as ethics-based judgments. While current approaches have focused on fine-tuning LLMs with curated datasets to improve their capabilities on such tasks, choosing the optimal learning paradigm to enhance the ethical responses of LLMs remains an open research debate. In this work, we aim to address this fundamental question: can current learning paradigms enable LLMs to acquire sufficient moral reasoning capabilities? Drawing from distributional semantics theory and the pragmatic nature of moral discourse, our analysis indicates that performance improvements follow a mechanism similar to that of semantic-level tasks, and therefore remain affected by the pragmatic nature of morals latent in discourse, a phenomenon we name the pragmatic dilemma. We conclude that this pragmatic dilemma imposes significant limitations on the generalization ability of current learning paradigms, making it the primary bottleneck for moral reasoning acquisition in LLMs.