# From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs

arXiv: https://arxiv.org/abs/2609.28942 · HTML: https://mlsanity.com/paper/2609.28942/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 50 of its 72 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 30 of 50)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [KTO: Model Alignment as Prospect Theoretic Optimization](https://mlsanity.com/paper/2402.01306/) (2024)
- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Learning Bayesian Networks: The Combination of Knowledge and Statistical Data](https://mlsanity.com/paper/1302.6815/) (2013)
- [Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning](https://mlsanity.com/paper/2408.10075/) (2024)
- [Personalized Language Modeling from Personalized Human Feedback](https://mlsanity.com/paper/2402.05133/) (2024)
- [Decoding-Time Language Model Alignment with Multiple Objectives](https://mlsanity.com/paper/2406.18853/) (2024)
- [Deep Reinforcement Learning for Multi-objective Optimization](https://mlsanity.com/paper/1906.02386/) (2019)
- [PAD: Personalized Alignment of LLMs at Decoding-Time](https://mlsanity.com/paper/2410.04070/) (2024)
- [DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life](https://mlsanity.com/paper/2410.02683/) (2024)
- [From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models](https://mlsanity.com/paper/2308.12014/) (2023)
- [MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models](https://mlsanity.com/paper/2403.17141/) (2024)
- [Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens](https://mlsanity.com/paper/2505.18237/) (2025)
- [From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment](https://mlsanity.com/paper/2503.15463/) (2025)
- [SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs](https://mlsanity.com/paper/2506.05598/) (2025)
- [Aligning Language Models with Human Preferences via a Bayesian Approach](https://mlsanity.com/paper/2310.05782/) (2023)
- [Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment](https://mlsanity.com/paper/2505.15456/) (2025)
- [Enough Coin Flips Can Make LLMs Act Bayesian](https://mlsanity.com/paper/2503.04722/) (2025)
- [R2MED: A Benchmark for Reasoning-Driven Medical Retrieval](https://mlsanity.com/paper/2505.14558/) (2025)

## Abstract (arXiv)

Personalized value alignment has become increasingly important as large language models (LLMs) are expected to accommodate diverse user preferences. However, existing methods typically align model outputs with a static value profile across prompts, overlooking that the salience of value dimensions varies substantially across contexts. Inspired by Lewin's Field Theory, which views human behavior as jointly shaped by personal dispositions and situational constraints, we model personal values as priors and context-dependent preferences as posteriors. We propose BaCVA, an inference-time Bayesian Context-aware personalized Value Alignment method that approximates posterior personalized preferences by integrating static personal values with scenario-specific value salience. BaCVA first estimates contextual value salience from generally normative responses, and then employs a dual-view personalization module to infer posterior preferences from complementary personal-value and scenario-driven perspectives. This Bayesian formulation enables more accurate and adaptive personalized value alignment while improving data efficiency via prior values. Extensive experiments on benchmarks demonstrate its superiority over strong baselines.