# Value Alignment Tax: Measuring Value Trade-offs in LLM Alignment

arXiv: https://arxiv.org/abs/2602.12134 · HTML: https://mlsanity.com/paper/2602.12134/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 14 of 14)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Interpretable Deep Learning under Fire](https://mlsanity.com/paper/1812.00891/) (2018)
- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?](https://mlsanity.com/paper/2501.15463/) (2025)
- [Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement](https://mlsanity.com/paper/2505.08245/) (2025)
- [Can Language Models Reason about Individualistic Human Values and Preferences?](https://mlsanity.com/paper/2410.03868/) (2024)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing](https://mlsanity.com/paper/2406.14230/) (2024)
- [Improving fairness in speaker verification via Group-adapted Fusion Network](https://mlsanity.com/paper/2202.11323/) (2022)
- [Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights](https://mlsanity.com/paper/2506.06404/) (2025)
- [The power of moral words: Loaded language generates framing effects in the extreme dictator game](https://mlsanity.com/paper/1901.02314/) (2019)
- [Cultural Value Alignment in Large Language Models: A Prompt-based Analysis of Schwartz Values in Gemini, ChatGPT, and DeepSeek](https://mlsanity.com/paper/2505.17112/) (2025)

## Built on by (most cited first, 1 of 1)

- [Alignment as Iatrogenesis: Pastoral Power, Collective Pathology, and the Structural Limits of Monolingual Safety Evaluation](https://mlsanity.com/paper/2603.08723/) (2026)

## Abstract (arXiv)

Existing work on value alignment typically characterizes value relations statically, ignoring how alignment interventions, such as prompting, fine-tuning, or preference optimization, reshape the broader value system. In practice, aligning a target value can implicitly shift other values, creating value trade-offs that remain largely unmeasured. We introduce VAT, a framework that quantifies value trade-offs by measuring how alignment-induced changes propagate across interconnected values relative to achieved on-target gain. VAT captures the system-level dynamics of value expression under alignment intervention, enabling evaluation of both intended improvements and unintended side effects. Using a controlled scenario-action dataset grounded in Schwartz value theory, we collect paired pre-post normative judgments and analyze alignment effects across models, values, and interventions. Results show that alignment often produces uneven and structured co-movement among values, revealing systematic trade-offs between target and non-target values. These effects are largely invisible under conventional target-only evaluation, but become evident via VAT, highlighting process-level alignment risks and offering new insights into the dynamic nature of value alignment in LLMs. Dataset and code are open-sourced.