Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment
Not on the map. Why?
Abstract (arXiv)
Language models increasingly act as agents. An agent that says an action is wrong and then takes it anyway is a different failure from one that does not know better, and evaluations of stated values cannot see it. We build a pre-registered panel of 248 scenarios across five kinds of pressure. Each scenario is posed twice to the same model, once as the agent choosing what to do and once in the third person asking which option is right, so the model's own judgment is the reference. Every scenario has a twin with the pressure removed, and every model gets a positive control in which its operator orders the violating action, so that a missing gap can be told apart from a blind instrument. On OLMo-3-7B-Instruct, the model takes the action it judged wrong on about one in five pressuring scenarios, more often than on the same scenarios with the pressure removed. Across four instruct models the gap depends on the post-training recipe: OLMo-3 and Meta's Llama-3.1-8B-Instruct carry it; Tulu 3 shows none on the whole panel (above about 0.01 in probability) or on its own most-pressuring scenarios; Qwen2.5-7B-Instruct shows none on the whole panel (above about 0.02) and is unresolved on its own (0.083, -0.028 to 0.195). Meta's recipe and Ai2's Tulu 3 start from the same Llama-3.1 weights, and only Meta's carries the gap. Reading a chat model outside its chat template reverses the sign of its gap with nothing at stake (-0.038 against +0.055 under the template on OLMo-3), a distortion present on two of three recipes. On both models that carry it, reasoning about the stakes before acting moves the choice back toward the model's own judgment, against a same-length non-moral task, with or without the pressure; on OLMo-3, naming the norm at stake does about a third of that. The gap is a measurable target for post-training recipes, not a fixed property of pretrained weights.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 6 Oct 2026.
Builds on
21 of 24 references.
References · 21
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2024Alignment faking in large language modelscited by 360
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
- 2025Olmo 3cited by 129
- 2025Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?cited by 51
9 more
- 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 41
- 2026Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationscited by 17
- 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 15
- 2025Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?cited by 10
- 2026Knowing But Not Doing: Convergent Morality and Divergent Action in LLMscited by 7
- 2026Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Usecited by 7
- 2026The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisionscited by 3
- 2026Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actionscited by 1
- 2026Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Familiescited by 1
Built on by
Not cited yet.