# Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective

arXiv: https://arxiv.org/abs/2501.00581 · HTML: https://mlsanity.com/paper/2501.00581/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 31 Dec 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making](https://mlsanity.com/paper/2411.03865/) (2024)
- [Incorporating Pragmatic Reasoning Communication into Emergent Language](https://mlsanity.com/paper/2006.04109/) (2020)
- [Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning](https://mlsanity.com/paper/2406.08002/) (2024)
- [Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games](https://mlsanity.com/paper/2410.07863/) (2024)

## Built on by (most cited first, 9 of 9)

- [Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs](https://mlsanity.com/paper/2508.19594/) (2025)
- [AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference](https://mlsanity.com/paper/2505.13531/) (2025)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment](https://mlsanity.com/paper/2510.13387/) (2025)
- [VISPA: Pluralistic Alignment via Automatic Value Selection and Activation](https://mlsanity.com/paper/2601.12758/) (2026)
- [Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph](https://mlsanity.com/paper/2603.15527/) (2026)
- [BACH-V: Bridging Abstract and Concrete Human-Values in Large Language Models](https://mlsanity.com/paper/2601.14007/) (2026)
- [Bridging Values and Behavior: A Hierarchical Framework for Proactive Embodied Agents](https://mlsanity.com/paper/2604.27699/) (2026)
- [Steering Geometry: Validating Human Value Geometry in LLM Steering Space](https://mlsanity.com/paper/2609.06289/) (2026)

## Abstract (arXiv)

As large language models (LLMs) become increasingly integrated into critical applications, aligning their behavior with human values presents significant challenges. Current methods, such as Reinforcement Learning from Human Feedback (RLHF), typically focus on a limited set of coarse-grained values and are resource-intensive. Moreover, the correlations between these values remain implicit, leading to unclear explanations for value-steering outcomes. Our work argues that a latent causal value graph underlies the value dimensions of LLMs and that, despite alignment training, this structure remains significantly different from human value systems. We leverage these causal value graphs to guide two lightweight value-steering methods: role-based prompting and sparse autoencoder (SAE) steering, effectively mitigating unexpected side effects. Furthermore, SAE provides a more fine-grained approach to value steering. Experiments on Gemma-2B-IT and Llama3-8B-IT demonstrate the effectiveness and controllability of our methods.