Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs
Not on the map. Why?
Abstract (arXiv)
Human communities are governed by normative systems: shared standards that produce textitnorms dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by the fact that norms are vast in number, change quickly, and are often arbitrary (e.g., dress or language conventions). Thus, alignment requires textitnormative competence: the ability to discern from interaction alone what norms a community enforces without relying on static pretrained knowledge. We introduce a multi-agent community debate setting, where access to debate is governed by synthetic norms, to study normative competence in isolation from pretraining exposure. We show that baseline LLM agents fail to learn norms even when doing so would improve their accuracy. We then experiment with various textitnormative modules -- architectural components for norm inference -- finding that norm-following is highly sensitive to both the style of norm and the model powering the normative module, suggesting a lack of generalizability. Furthermore, when idiosyncratic, non-normative behaviors accompany the true norm, LLM agents exhibit an unselective attribution failure: they indiscriminately copy idiosyncratic noise alongside enforced rules, a pattern that persists even when imitating unnecessary behaviors is explicitly penalized. To the best of our knowledge, our work is the first to operationalize and evaluate normative competence in LLMs, demonstrating that current AI systems excel at behavioral mimicry but lack the capacity to discern socially enforced order.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 50 references that are in the map. First posted on arXiv 7 Oct 2026.
Builds on
23 of 50 references.
References · 23
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2017Safe Reinforcement Learning via Shieldingcited by 427
- 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
- 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
- 2024A Roadmap to Pluralistic Alignmentcited by 175
- 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
11 more
- 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
- 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
- 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 90
- 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
- 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 63
- 2023Contrastive Chain-of-Thought Promptingcited by 52
- 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
- 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
- 2018Legible Normativity for AI Alignment: The Value of Silly Rulescited by 15
- 2026A Theory of Appropriateness That Accounts for Norms of Rationalitycited by 4
- 2025Controlling the Risk of Corrupted Contexts for Language Models via Early-Exitingcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Harnessing the power of LLMs for normative reasoning in MASs
- 2026Normative Common Ground Replication (NormCoRe): Replication-by-Translation for Studying Norms in Multi-Agent AI
- 2024Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study
- 2024Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation
- 2024Evolution of Social Norms in LLM Agents using Natural Language
- 2025Normative Conflicts and Shallow AI Alignment
- 2026Social Norm Reasoning in Multimodal Language Models: An Evaluation
- 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation