PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Large language models are being extensively used to simulate individual user behavior, yet faithfully representing a population requires capturing the systematic variation in values, beliefs, and cultural norms that distinguish one group from another. We introduce Population Aligned Language Models (PALMs), a suite of models each aligned to specific populations, covering five countries: USA, India, Brazil, France and Italy. PALMs are created by synthesizing rationales grounded in psychological and cultural constructs and using these as latent supervision during preference tuning for population-specific alignment. Evaluated across four dimensions: personality, values and beliefs, cultural norms, and morality, PALMs consistently outperform baselines, including culture-specialized models, achieving an average of 8.59% relative improvement over the best baseline across all five populations. Notably, construct-grounded rationales outperform both demographic prompting and survey-based fine-tuning, suggesting that grounding preference learning in psychology and culture provides a richer inductive signal than surface-level response distributions. We further demonstrate strong generalization to downstream applications with- out task-specific supervision: outperforming best baselines by 5.19% in personalized reward modeling, 6.34% in population simulation, and showing strong transfer to social reasoning tasks. Datasets and code are available at: https://github.com/limenlp/PALMs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 61 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
33 of 61 references.
Closest ancestors · 5
Same direction, newest first
- 2026Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization8 Apr
- 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset13 Jul
- 2024Benchmarking Distributional Alignment of Large Language Models8 Nov
- 2024Investigating Cultural Alignment of Large Language Models20 Feb
- 2023Aligning Language Models to User Opinions24 May
Other references · 28
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Let's Verify Step by Stepcited by 3,598
- 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2022Using cognitive psychology to understand GPT-3cited by 389
- 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
- 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
- 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
- 2018Training Millions of Personalized Dialogue Agentscited by 126
- 2023Generative Language Models Exhibit Social Identity Biasescited by 107
16 more
- 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
- 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
- 2019Personalizing Dialogue Agents via Meta-Learningcited by 83
- 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 64
- 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
- 2023The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographicscited by 45
- 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
- 2024An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevancecited by 36
- 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 17
- 2025Improving Language Model Personas via Rationalization with Psychological Scaffoldscited by 9
- 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignmentcited by 9
- 2024Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensivenesscited by 7
- 2025Should you use LLMs to simulate opinions? Quality checks for early-stage deliberationcited by 6
- 2025Can LLMs Grasp Implicit Cultural Values? Benchmarking LLMs' Cultural Intelligence with CQ-Benchcited by 3
- 2025GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferencescited by 3
- 2025PrimeX: A Dataset of Worldview, Opinion, and Explanationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
- 2026DiSCo: A Distribution-First Steering and Cultural Prior Evaluation Framework for Measuring Cultural Preference Bias in LLMs
- 2026Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment
- 2026Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
- 2026Unpacking Human Preference for LLMs: Demographically Aware Evaluation with the HUMAINE Framework
- 2024Using LLMs to Model the Beliefs and Preferences of Targeted Populations
- 2024Towards a Unified View of Preference Learning for Large Language Models: A Survey
- 2024A Survey on Human Preference Learning for Large Language Models