# Demonstrations of the Potential of AI-based Political Issue Polling

arXiv: https://arxiv.org/abs/2307.04781 · HTML: https://mlsanity.com/paper/2307.04781/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2023.

## Built on by (most cited first, 14 of 14)

- [Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information](https://mlsanity.com/paper/2402.18144/) (2024)
- [Evaluating language models as risk scores](https://mlsanity.com/paper/2407.14614/) (2024)
- [LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra](https://mlsanity.com/paper/2507.15815/) (2025)
- [The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models](https://mlsanity.com/paper/2406.11096/) (2024)
- [Are Large Language Models Chameleons? An Attempt to Simulate Social Surveys](https://mlsanity.com/paper/2405.19323/) (2024)
- [How will advanced AI systems impact democracy?](https://mlsanity.com/paper/2409.06729/) (2024)
- [United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections](https://mlsanity.com/paper/2409.09045/) (2024)
- [Large Language Models in Politics and Democracy: A Comprehensive Survey](https://mlsanity.com/paper/2412.04498/) (2024)
- [On the Credibility of Evaluating LLMs using Survey Questions](https://mlsanity.com/paper/2602.04033/) (2026)
- [Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases](https://mlsanity.com/paper/2502.18282/) (2025)
- [When Persona Attributes Improve Population Alignment in Large Language Models](https://mlsanity.com/paper/2609.02526/) (2026)
- [Characterizing the ability of LLMs to recapitulate Americans' distributional responses to public opinion polling questions across political issues](https://mlsanity.com/paper/2603.20229/) (2026)
- [Aplaud: Adaptive Personalized Low-Rank Decomposition for User-Specific LLM](https://mlsanity.com/paper/2609.04738/) (2026)
- [Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning](https://mlsanity.com/paper/2609.04485/) (2026)

## Abstract (arXiv)

Political polling is a multi-billion dollar industry with outsized influence on the societal trajectory of the United States and nations around the world. However, it has been challenged by factors that stress its cost, availability, and accuracy. At the same time, artificial intelligence (AI) chatbots have become compelling stand-ins for human behavior, powered by increasingly sophisticated large language models (LLMs). Could AI chatbots be an effective tool for anticipating public opinion on controversial issues to the extent that they could be used by campaigns, interest groups, and polling firms? We have developed a prompt engineering methodology for eliciting human-like survey responses from ChatGPT, which simulate the response to a policy question of a person described by a set of demographic factors, and produce both an ordinal numeric response score and a textual justification. We execute large scale experiments, querying for thousands of simulated responses at a cost far lower than human surveys. We compare simulated data to human issue polling data from the Cooperative Election Study (CES). We find that ChatGPT is effective at anticipating both the mean level and distribution of public opinion on a variety of policy issues such as abortion bans and approval of the US Supreme Court, particularly in their ideological breakdown (correlation typically >85%). However, it is less successful at anticipating demographic-level differences. Moreover, ChatGPT tends to overgeneralize to new policy issues that arose after its training data was collected, such as US support for involvement in the war in Ukraine. Our work has implications for our understanding of the strengths and limitations of the current generation of AI chatbots as virtual publics or online listening platforms, future directions for LLM development, and applications of AI tools to the political domain. (Abridged)