mlsanityData to 7 Oct 2026

Whose Opinions Do Language Models Reflect?

arXiv 2303.17548 ↗First posted 30 Mar 2023cs.CL cs.AI cs.CYcited by 759

Not on the map. Why?

Cited by 133 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals). Our code and data are available at https://github.com/tatsu-lab/opinions_qa.

Key facts. Cited by 759 papers in the map (data through 7 Oct 2026). Builds on 27 of its 68 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

27 of 68 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2022Red Teaming Language Models with Language Modelscited by 983
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  11. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  12. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
15 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  2. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  3. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  4. 2022Using cognitive psychology to understand GPT-3cited by 389
  5. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  6. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  7. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  8. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  9. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  10. 2021TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generationcited by 118
  11. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  12. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 100
  13. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 71
  14. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  15. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 75; 2024: 169; 2025: 217; 2026: 298.

759 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  3. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  4. 2023How is ChatGPT's behavior changing over time?cited by 265
  5. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  6. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  7. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  8. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  9. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  10. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  11. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  12. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
38 more
  1. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  2. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  3. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  4. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  5. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  6. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  7. 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
  8. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  9. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  10. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
  11. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  12. 2024Personalized Language Modeling from Personalized Human Feedbackcited by 124
  13. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  14. 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 121
  15. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 120
  16. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  17. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 112
  18. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  19. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  20. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  21. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  22. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  23. 2024Can LLM be a Personalized Judge?cited by 99
  24. 2023Aligning Language Models to User Opinionscited by 98
  25. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  26. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  27. 2025LLM Social Simulations Are a Promising Research Methodcited by 97
  28. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  29. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  30. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  31. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 80
  32. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  33. 2024Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooMcited by 77
  34. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  35. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 75
  36. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
  37. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  38. 2024On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularizationcited by 72

50 most cited of 759 All 759 (JSON)