# Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble

arXiv: https://arxiv.org/abs/2509.11311 · HTML: https://mlsanity.com/paper/2509.11311/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 38 of its 69 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 30 of 38)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Selection via Proxy: Efficient Data Selection for Deep Learning](https://mlsanity.com/paper/1906.11829/) (2019)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration](https://mlsanity.com/paper/2406.15951/) (2024)
- [Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties](https://mlsanity.com/paper/2309.00779/) (2023)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [Aligning to Thousands of Preferences via System Message Generalization](https://mlsanity.com/paper/2405.17977/) (2024)
- [Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity](https://mlsanity.com/paper/2209.12106/) (2022)
- [PERSONA: A Reproducible Testbed for Pluralistic Alignment](https://mlsanity.com/paper/2407.17387/) (2024)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents](https://mlsanity.com/paper/2310.17512/) (2023)
- [Group Preference Optimization: Few-Shot Alignment of Large Language Models](https://mlsanity.com/paper/2310.11523/) (2023)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations](https://mlsanity.com/paper/2502.07068/) (2025)
- [Hidden Incentives for Auto-Induced Distributional Shift](https://mlsanity.com/paper/2009.09153/) (2020)
- [Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs](https://mlsanity.com/paper/2410.08020/) (2024)
- [From Distributional to Overton Pluralism: Investigating Large Language Model Alignment](https://mlsanity.com/paper/2406.17692/) (2024)
- [Cultural Palette: Pluralising Culture Alignment via Multi-agent Palette](https://mlsanity.com/paper/2412.11167/) (2024)

## Built on by (most cited first, 2 of 2)

- [Adaptive Querying with AI Persona Priors](https://mlsanity.com/paper/2605.00696/) (2026)
- [Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions](https://mlsanity.com/paper/2609.38555/) (2026)

## Abstract (arXiv)

Large language models are increasingly used as proxies for human subjects in social science research, yet external validity requires that synthetic agents faithfully reflect the preferences of target human populations. We introduce *preference reconstruction theory*, a framework that formalizes preference alignment as a representation learning problem: constructing a functional basis of proxy agents and recovering population preferences through weighted aggregation. We implement this via *Prompts to Proxies* (textttP2P), a modular two-stage system. Stage 1 uses structured prompting with entropy-based adaptive sampling to construct a diverse agent pool spanning the latent preference space. Stage 2 employs L1-regularized regression to select a compact ensemble whose aggregate response distributions align with observed data from the target population. textttP2P requires no finetuning and no access to sensitive demographic data, incurring only API inference costs. We validate the approach on 14 waves of the American Trends Panel, achieving an average test MSE of 0.014 across diverse topics at approximately 0.8 USD per survey. We additionally test it on the World Values Survey, demonstrating its potential to generalize across locales. When stress-tested against an SFT-aligned baseline, textttP2P achieves competitive performance using less than 3% of the training data.