The Profit Alignment Problem: How Profit Mandates Induce Alignment Failures in LLMs
Not on the map. Why?
Abstract (arXiv)
We show that ordinary business language --- "maximize profitability" --- induces profit-oriented ambiguity resolution: LLMs systematically dismiss ambiguous signals of potential safety violations to serve business objectives. In 3,600 controlled trials across eight reasoning-capable LLMs, adding a profit mandate to otherwise identical prompts increases risk-dismissing judgments by 6.8 percentage points (p < 0.0001), suppresses board escalation recommendations by 13.9pp (p < 0.0001), and shifts severity assessments downward (p < 0.0001). The mandate never instructs models to downplay risks; instead, chain-of-thought traces reveal motivated reasoning: models acknowledge concerns, then invoke profit logic to justify dismissing them. We characterize these findings as the Profit Alignment Problem: when AI systems are given ordinary business objectives, they develop systematic strategies for suppressing inconvenient information that no designer intended or specified.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 8 of its 15 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
8 of 15 references.
References · 8
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2024Alignment faking in large language modelscited by 360
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Chat Bankman-Fried: an Exploration of LLM Alignment in Finance
- 2026In LLM Reasoning, there is Irrationality on top of Value Misalignment
- 2024AI as Decision-Maker: Ethics and Risk Preferences of LLMs
- 2026Probing Outcome-Level Resemblance and Mechanism-Level Alignment in LLM Risk Decisions: Evidence from the St. Petersburg Game
- 2023Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models
- 2026COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMs
- 2025Evaluating and Aligning Human Economic Risk Preferences in LLMs
- 2026GAIN: A Benchmark for Goal-Aligned Decision-Making of Large Language Models under Imperfect Norms