AI as Decision-Maker: Ethics and Risk Preferences of LLMs
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Large Language Models (LLMs) exhibit surprisingly diverse risk preferences when acting as AI decision makers, a crucial characteristic whose origins remain poorly understood despite their expanding economic roles. We analyze 50 LLMs using behavioral tasks, finding stable but diverse risk profiles. Alignment tuning for harmlessness, helpfulness, and honesty significantly increases risk aversion, causally increasing risk aversion confirmed via comparative difference analysis: a ten percent ethics increase cuts risk appetite two to eight percent. This induced caution persists against prompts and affects economic forecasts. Alignment enhances safety but may also suppress valuable risk taking, revealing a tradeoff risking suboptimal economic outcomes. With AI models becoming more powerful and influential in economic decisions while alignment grows increasingly critical, our empirical framework serves as an adaptable and enduring benchmark to track risk preferences and monitor this crucial tension between ethical alignment and economically valuable risk-taking.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. First posted on arXiv 3 Jun 2024.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2024: 2; 2025: 1; 2026: 4.
7 papers.
- 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendationscited by 33
- 2024Uncertainty Quantification in Large Language Models Through Convex Hull Analysiscited by 12
- 2026RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?cited by 9
- 2026Can Revealed Preferences Clarify LLM Alignment and Steering?cited by 4
- 2025Simulating Macroeconomic Expectations in Survey Experiments with LLM-based Economic Agentscited by 1
- 2026Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocationcited by 1
- 2026High-Stakes Decisions with Language Models: Insights from Emergency Triagecited by 0
- 2026High-Stakes Decisions with Language Models: Insights from Emergency Triage2 Aug 2026
- 2026Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation1 Jun 2026
- 2026Can Revealed Preferences Clarify LLM Alignment and Steering?8 May 2026
- 2026RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?6 Feb 2026
- 2025Simulating Macroeconomic Expectations in Survey Experiments with LLM-based Economic Agents23 May 2025
- 2024Uncertainty Quantification in Large Language Models Through Convex Hull Analysis28 Jun 2024
- 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations18 Jun 2024
Similar, not cited
Close in wording, no citation either way
- 2025Evaluating and Aligning Human Economic Risk Preferences in LLMs
- 2024Chat Bankman-Fried: an Exploration of LLM Alignment in Finance
- 2025Risk Profiling and Modulation for LLMs
- 2024GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning
- 2023Unpacking the Ethical Value Alignment in Big Models
- 2024Strong Preferences Affect the Robustness of Preference Models and Value Alignment
- 2025Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices
- 2025Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking