# Language Models Trained on Media Diets Can Predict Public Opinion

arXiv: https://arxiv.org/abs/2303.16779 · HTML: https://mlsanity.com/paper/2303.16779/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 67 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://mlsanity.com/paper/2104.08315/) (2021)
- [Linking artificial and human neural representations of language](https://mlsanity.com/paper/1910.01244/) (2019)
- [The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter](https://mlsanity.com/paper/1405.1438/) (2014)

## Built on by (most cited first, 30 of 30)

- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [Virtual Personas for Language Models via an Anthology of Backstories](https://mlsanity.com/paper/2407.06576/) (2024)
- [Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information](https://mlsanity.com/paper/2402.18144/) (2024)
- [Opportunities and Risks of LLMs for Scalable Deliberation with Polis](https://mlsanity.com/paper/2306.11932/) (2023)
- [Finetuning LLMs for Human Behavior Prediction in Social Science Experiments](https://mlsanity.com/paper/2509.05830/) (2025)
- [Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas](https://mlsanity.com/paper/2408.06929/) (2024)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look](https://mlsanity.com/paper/2411.08275/) (2024)
- [Improving Interpersonal Communication by Simulating Audiences with Language Models](https://mlsanity.com/paper/2311.00687/) (2023)
- [Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion](https://mlsanity.com/paper/2407.08563/) (2024)
- [COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities](https://mlsanity.com/paper/2406.12074/) (2024)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Reading Between the Tweets: Deciphering Ideological Stances of Interconnected Mixed-Ideology Communities](https://mlsanity.com/paper/2402.01091/) (2024)
- [Can LLMs Learn Macroeconomic Narratives from Social Media?](https://mlsanity.com/paper/2406.12109/) (2024)
- [United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections](https://mlsanity.com/paper/2409.09045/) (2024)
- [Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans](https://mlsanity.com/paper/2601.16355/) (2026)
- [Assessing Generalization for Subpopulation Representative Modeling via In-Context Learning](https://mlsanity.com/paper/2402.07368/) (2024)
- [Learning from Convenience Samples: A Case Study on Fine-Tuning LLMs for Survey Non-response in the German Longitudinal Election Study](https://mlsanity.com/paper/2509.25063/) (2025)
- [From Demographics to Survey Anchors: Evaluating LLM Agents for Modeling Retirement Attitudes](https://mlsanity.com/paper/2605.16303/) (2026)
- [Graph-Based Alternatives to LLMs for Human Simulation](https://mlsanity.com/paper/2511.02135/) (2025)
- [A Conceptual Model for Attributions in Event-Centric Knowledge Graphs](https://mlsanity.com/paper/2503.03563/) (2025)
- [Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods](https://mlsanity.com/paper/2505.19402/) (2025)
- [What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?](https://mlsanity.com/paper/2511.18616/) (2025)
- [Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task](https://mlsanity.com/paper/2603.03295/) (2026)
- [Understanding Reliability in LLM-based Human Behavior Simulation](https://mlsanity.com/paper/2609.25066/) (2026)

## Abstract (arXiv)

Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, TV broadcast, or radio show content -- that can emulate the opinions of subpopulations that have consumed a set of media. To validate this method, we use as ground truth the opinions expressed in U.S. nationally representative surveys on COVID-19 and consumer confidence. Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions and robust to phrasing and channels of media exposure, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption. Probing language models provides a powerful new method for investigating media effects, has practical applications in supplementing polls and forecasting public opinion, and suggests a need for further study of the surprising fidelity with which neural language models can predict human responses.