# Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs

arXiv: https://arxiv.org/abs/2310.07251 · HTML: https://mlsanity.com/paper/2310.07251/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2023.

## Built on by (most cited first, 30 of 64)

- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models](https://mlsanity.com/paper/2404.12464/) (2024)
- [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024)
- [PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data](https://mlsanity.com/paper/2406.15053/) (2024)
- [CulturePark: Boosting Cross-cultural Understanding in Large Language Models](https://mlsanity.com/paper/2405.15145/) (2024)
- [The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process](https://mlsanity.com/paper/2506.01080/) (2025)
- [AI Alignment at Your Discretion](https://mlsanity.com/paper/2502.10441/) (2025)
- [The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models](https://mlsanity.com/paper/2406.11096/) (2024)
- [Agentic AI Needs a Systems Theory](https://mlsanity.com/paper/2503.00237/) (2025)
- [Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classification](https://mlsanity.com/paper/2402.18502/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Unmasking Conversational Bias in AI Multiagent Systems](https://mlsanity.com/paper/2501.14844/) (2025)
- [From Melting Pots to Misrepresentations: Exploring Harms in Generative AI](https://mlsanity.com/paper/2403.10776/) (2024)
- [The Confidence Paradox: Can LLM Know When It's Wrong](https://mlsanity.com/paper/2506.23464/) (2025)
- [Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs](https://mlsanity.com/paper/2506.13082/) (2025)
- [Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test](https://mlsanity.com/paper/2402.02135/) (2024)
- [Surprising gender biases in GPT](https://mlsanity.com/paper/2407.06003/) (2024)
- [Meta-Cultural Competence: Climbing the Right Hill of Cultural Awareness](https://mlsanity.com/paper/2502.09637/) (2025)
- [Societal Alignment Frameworks Can Improve LLM Alignment](https://mlsanity.com/paper/2503.00069/) (2025)
- [ProRefine: Inference-Time Prompt Refinement with Textual Feedback](https://mlsanity.com/paper/2506.05305/) (2025)
- [Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation](https://mlsanity.com/paper/2506.19352/) (2025)
- [The Fragility Of Moral Judgment In Large Language Models](https://mlsanity.com/paper/2603.05651/) (2026)
- [Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy](https://mlsanity.com/paper/2411.06723/) (2024)
- [One world, one opinion? The superstar effect in LLM responses](https://mlsanity.com/paper/2412.10281/) (2024)
- [Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning](https://mlsanity.com/paper/2503.11127/) (2025)
- [Multilingual Amnesia: On the Transferability of Unlearning in Multilingual LLMs](https://mlsanity.com/paper/2601.05641/) (2026)
- [Towards Geo-Culturally Grounded LLM Generations](https://mlsanity.com/paper/2502.13497/) (2025)
- [Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models](https://mlsanity.com/paper/2508.17184/) (2025)
- [Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities](https://mlsanity.com/paper/2406.01757/) (2024)

## Abstract (arXiv)

In this position paper, we argue that instead of morally aligning LLMs to specific set of ethical principles, we should infuse generic ethical reasoning capabilities into them so that they can handle value pluralism at a global scale. When provided with an ethical policy, an LLM should be capable of making decisions that are ethically consistent to the policy. We develop a framework that integrates moral dilemmas with moral principles pertaining to different foramlisms of normative ethics, and at different levels of abstractions. Initial experiments with GPT-x models shows that while GPT-4 is a nearly perfect ethical reasoner, the models still have bias towards the moral values of Western and English speaking societies.