# Llms, Virtual Users, and Bias: Predicting Any Survey Question Without Human Data

arXiv: https://arxiv.org/abs/2503.16498 · HTML: https://mlsanity.com/paper/2503.16498/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 11 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 4 of 4)

- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction](https://mlsanity.com/paper/2305.09620/) (2023)

## Built on by (most cited first, 3 of 3)

- [Exact Synthetic Populations for Scalable Societal and Market Modeling](https://mlsanity.com/paper/2512.07306/) (2025)
- [How Large Language Models Systematically Misrepresent American Climate Opinions](https://mlsanity.com/paper/2512.23889/) (2025)
- [Generating Public Health Responses using Survey-Augmented Large Language Models](https://mlsanity.com/paper/2606.21820/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) offer a promising alternative to traditional survey methods, potentially enhancing efficiency and reducing costs. In this study, we use LLMs to create virtual populations that answer survey questions, enabling us to predict outcomes comparable to human responses. We evaluate several LLMs-including GPT-4o, GPT-3.5, Claude 3.5-Sonnet, and versions of the Llama and Mistral models-comparing their performance to that of a traditional Random Forests algorithm using demographic data from the World Values Survey (WVS). LLMs demonstrate competitive performance overall, with the significant advantage of requiring no additional training data. However, they exhibit biases when predicting responses for certain religious and population groups, underperforming in these areas. On the other hand, Random Forests demonstrate stronger performance than LLMs when trained with sufficient data. We observe that removing censorship mechanisms from LLMs significantly improves predictive accuracy, particularly for underrepresented demographic segments where censored models struggle. These findings highlight the importance of addressing biases and reconsidering censorship approaches in LLMs to enhance their reliability and fairness in public opinion research.