# What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?

arXiv: https://arxiv.org/abs/2511.18616 · HTML: https://mlsanity.com/paper/2511.18616/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 23 Nov 2025.

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [The Geometry of Culture: Analyzing Meaning through Word Embeddings](https://mlsanity.com/paper/1803.09288/) (2018)
- [LLM-Rec: Personalized Recommendation via Prompting Large Language Models](https://mlsanity.com/paper/2307.15780/) (2023)
- [CommunityLM: Probing Partisan Worldviews from Language Models](https://mlsanity.com/paper/2209.07065/) (2022)
- [Language Models Trained on Media Diets Can Predict Public Opinion](https://mlsanity.com/paper/2303.16779/) (2023)
- [DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines](https://mlsanity.com/paper/2312.13382/) (2023)
- [A Corpus for Modeling User and Language Effects in Argumentation on Online Debating](https://mlsanity.com/paper/1906.11310/) (2019)
- [Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science](https://mlsanity.com/paper/2305.14310/) (2023)
- [Representation Bias in Political Sample Simulations with Large Language Models](https://mlsanity.com/paper/2407.11409/) (2024)
- [A semantic embedding space based on large language models for modelling human beliefs](https://mlsanity.com/paper/2408.07237/) (2024)
- [Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance](https://mlsanity.com/paper/2403.00236/) (2024)

## Abstract (arXiv)

Beliefs shape how people reason, communicate, and behave. Rather than existing in isolation, they exhibit a rich correlational structure--some connected through logical dependencies, others through indirect associations or social processes. As usage of large language models (LLMs) becomes more ubiquitous in our society, LLMs' ability to understand and reason through human beliefs has many implications from privacy issues to personalized persuasion and the potential for stereotyping. Yet how LLMs capture this interrelated landscape of beliefs remains unclear. For instance, when predicting someone's beliefs, what information affects the prediction most--who they are (demographics), what else they believe (prior stances), or a combination of both? We address these questions using data from an online debate platform, evaluating the ability of off-the-shelf open-weight LLMs to predict individuals' stance under four conditions: no context, demographics only, prior beliefs only, and both combined. We find that both types of information improve predictions over a blind baseline, with their combination yielding the best performance in most cases. However, the relative value of each varies substantially across belief domains. These findings reveal how current LLMs leverage different types of social information when reasoning about human beliefs, highlighting both their capabilities and limitations.