# Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs

arXiv: https://arxiv.org/abs/2610.10906 · HTML: https://mlsanity.com/paper/2610.10906/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models](https://mlsanity.com/paper/2404.16019/) (2024)
- [Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences](https://mlsanity.com/paper/2012.15738/) (2020)
- [Theory of Mind for Multi-Agent Collaboration via Large Language Models](https://mlsanity.com/paper/2310.10701/) (2023)
- [Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains](https://mlsanity.com/paper/2501.05707/) (2025)
- [Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback](https://mlsanity.com/paper/2303.05453/) (2023)
- [Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate](https://mlsanity.com/paper/2509.05396/) (2025)
- [Contrastive Chain-of-Thought Prompting](https://mlsanity.com/paper/2311.09277/) (2023)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [Few-shot Language Coordination by Modeling Theory of Mind](https://mlsanity.com/paper/2107.05697/) (2021)
- [Legible Normativity for AI Alignment: The Value of Silly Rules](https://mlsanity.com/paper/1811.01267/) (2018)
- [A Theory of Appropriateness That Accounts for Norms of Rationality](https://mlsanity.com/paper/2603.14050/) (2026)
- [Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting](https://mlsanity.com/paper/2510.02480/) (2025)

## Abstract (arXiv)

Human communities are governed by normative systems: shared standards that produce textitnorms dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by the fact that norms are vast in number, change quickly, and are often arbitrary (e.g., dress or language conventions). Thus, alignment requires textitnormative competence: the ability to discern from interaction alone what norms a community enforces without relying on static pretrained knowledge. We introduce a multi-agent community debate setting, where access to debate is governed by synthetic norms, to study normative competence in isolation from pretraining exposure. We show that baseline LLM agents fail to learn norms even when doing so would improve their accuracy. We then experiment with various textitnormative modules -- architectural components for norm inference -- finding that norm-following is highly sensitive to both the style of norm and the model powering the normative module, suggesting a lack of generalizability. Furthermore, when idiosyncratic, non-normative behaviors accompany the true norm, LLM agents exhibit an unselective attribution failure: they indiscriminately copy idiosyncratic noise alongside enforced rules, a pattern that persists even when imitating unnecessary behaviors is explicitly penalized. To the best of our knowledge, our work is the first to operationalize and evaluate normative competence in LLMs, demonstrating that current AI systems excel at behavioral mimicry but lack the capacity to discern socially enforced order.