mlsanityData to 7 Oct 2026

What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?

arXiv 2511.18616 ↗First posted 23 Nov 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Beliefs shape how people reason, communicate, and behave. Rather than existing in isolation, they exhibit a rich correlational structure--some connected through logical dependencies, others through indirect associations or social processes. As usage of large language models (LLMs) becomes more ubiquitous in our society, LLMs' ability to understand and reason through human beliefs has many implications from privacy issues to personalized persuasion and the potential for stereotyping. Yet how LLMs capture this interrelated landscape of beliefs remains unclear. For instance, when predicting someone's beliefs, what information affects the prediction most--who they are (demographics), what else they believe (prior stances), or a combination of both? We address these questions using data from an online debate platform, evaluating the ability of off-the-shelf open-weight LLMs to predict individuals' stance under four conditions: no context, demographics only, prior beliefs only, and both combined. We find that both types of information improve predictions over a blind baseline, with their combination yielding the best performance in most cases. However, the relative value of each varies substantially across belief domains. These findings reveal how current LLMs leverage different types of social information when reasoning about human beliefs, highlighting both their capabilities and limitations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

22 of 40 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  9. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  10. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  11. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
  12. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
10 more
  1. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  2. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  3. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
  4. 2023Language Models Trained on Media Diets Can Predict Public Opinioncited by 30
  5. 2023DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelinescited by 27
  6. 2019A Corpus for Modeling User and Language Effects in Argumentation on Online Debatingcited by 22
  7. 2023Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Sciencecited by 18
  8. 2024Representation Bias in Political Sample Simulations with Large Language Modelscited by 10
  9. 2024A semantic embedding space based on large language models for modelling human beliefscited by 9
  10. 2024Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performancecited by 5

Built on by

Not cited yet.