mlsanityData to 14 Oct 2026

How Language Models Organize and Structure Moral Knowledge

arXiv 2608.27402 ↗First posted 27 Aug 2026cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

How do large language models (LLMs) organize moral knowledge? Models detect moral content broadly, but detection is a low bar. We ask whether they go further, distinguishing moral foundations from one another and organizing the relationships between them geometrically. We train six independent linear probes on open-weight language models, one per Moral Foundations Theory (MFT) category (care/harm, fair/cheat, lib/oppress, loy/betray, auth/subv, sanc/degrade), and examine how the resulting directions relate to each other in representation space. We find the directions neither collapse into a single moral detector nor isolate from one another. Rather, they span a near-maximal number of independent dimensions while sharing a positive common component. The shared component is the signature of integration, and it is moral-specific relative to a matched non-moral concept battery built identically (mean pairwise cosine 0.26 vs. 0.013). The geometry is consistent across architectures and scale and reaches its integration regime early in pre-training, well before probe accuracy saturates. The structure the model discovers shows no evidence of the individualizing/binding distinction predicted by Moral Foundations Theory (an underpowered test: only 10 distinct splits exist, so it cannot reject at the 0.05 level) but rather reflects corpus statistics. Extending to moral dilemmas, each dilemma direction partially composes from its component foundations, at 2.7x a mismatched-pair baseline, while the majority of its variance encodes conflict-specific structure. The model represents moral tension itself, not a pre-resolved judgment.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 10 of its 16 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

10 of 16 references.

References · 10

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  3. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  5. 2020Aligning AI With Shared Human Valuescited by 723
  6. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  8. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 301
  9. 2026When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysiscited by 4
  10. 2026Output Dilution: Redundant but Fragile Representations in MoE Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tracing Moral Foundations in Large Language Models
  2. 2024The Moral Mind(s) of Large Language Models
  3. 2026Mechanistic Origin of Moral Indifference in Language Models
  4. 2026Representational alignment yields generalizable safety in language models
  5. 2025Differences in the Moral Foundations of Large Language Models
  6. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning
  7. 2025Diagnosing Moral Reasoning Acquisition in Language Models: Pragmatics and Generalization
  8. 2026Rule vs.Consequence: Dissociable Internal Representations of Moral Reasoning in LLMs