# Opportunities and Risks of LLMs for Scalable Deliberation with Polis

arXiv: https://arxiv.org/abs/2306.11932 · HTML: https://mlsanity.com/paper/2306.11932/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 86 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [Evaluating the Underlying Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.08783/) (2019)

## Built on by (most cited first, 30 of 42)

- [Generative Social Choice](https://mlsanity.com/paper/2309.01291/) (2023)
- [Emergence of polarization in networks of large language model agents](https://mlsanity.com/paper/2501.05171/) (2025)
- [Explaining Datasets in Words: Statistical Models with Natural Language Parameters](https://mlsanity.com/paper/2409.08466/) (2024)
- [Generative Social Choice: The Next Generation](https://mlsanity.com/paper/2505.22939/) (2025)
- [GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases](https://mlsanity.com/paper/2405.16205/) (2024)
- [Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey](https://mlsanity.com/paper/2503.01513/) (2025)
- [Generative AI voting: fair collective choice is resilient to LLM biases and inconsistencies](https://mlsanity.com/paper/2406.11871/) (2024)
- [Benchmarking Overton Pluralism in LLMs](https://mlsanity.com/paper/2512.01351/) (2025)
- [Leveraging Large Language Models for Collective Decision-Making](https://mlsanity.com/paper/2311.04928/) (2023)
- [AI and the Future of Digital Public Squares](https://mlsanity.com/paper/2412.09988/) (2024)
- [Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMs](https://mlsanity.com/paper/2510.05154/) (2025)
- [If the Sources Could Talk: Evaluating Large Language Models for Research Assistance in History](https://mlsanity.com/paper/2310.10808/) (2023)
- [How will advanced AI systems impact democracy?](https://mlsanity.com/paper/2409.06729/) (2024)
- [Artificial Intelligence in Deliberation: The AI Penalty and the Emergence of a New Deliberative Divide](https://mlsanity.com/paper/2503.07690/) (2025)
- [Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers](https://mlsanity.com/paper/2410.22937/) (2024)
- [LLMs as mediators: Can they diagnose conflicts accurately?](https://mlsanity.com/paper/2412.14675/) (2024)
- [Deliberate Lab: A Platform for Real-Time Human-AI Social Experiments](https://mlsanity.com/paper/2510.13011/) (2025)
- [Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitation](https://mlsanity.com/paper/2503.16505/) (2025)
- [PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives](https://mlsanity.com/paper/2505.19558/) (2025)
- [Question the Questions: Auditing Representation in Online Deliberative Processes](https://mlsanity.com/paper/2511.04588/) (2025)
- [The Variance Paradox: How AI Reduces Diversity but Increases Novelty](https://mlsanity.com/paper/2508.19264/) (2025)
- [An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies](https://mlsanity.com/paper/2509.12577/) (2025)
- [Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making](https://mlsanity.com/paper/2509.19643/) (2025)
- [Agora: Teaching the Skill of Consensus-Finding with AI Personas Grounded in Human Voice](https://mlsanity.com/paper/2603.07339/) (2026)
- [Finding Common Ground in a Sea of Alternatives](https://mlsanity.com/paper/2603.16751/) (2026)
- [Embeddings for Preferences, Not Semantics](https://mlsanity.com/paper/2605.08360/) (2026)
- [To Facilitate or not to Facilitate: Human and LLM Facilitator Tendencies in Online Discussions](https://mlsanity.com/paper/2607.28643/) (2026)
- [Estimating Contribution Quality in Online Deliberations Using a Large Language Model](https://mlsanity.com/paper/2408.11936/) (2024)
- [Algorithmic Shortlisting in Participatory Budgeting](https://mlsanity.com/paper/2508.06577/) (2025)
- [Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset](https://mlsanity.com/paper/2510.20508/) (2025)

## Abstract (arXiv)

Polis is a platform that leverages machine intelligence to scale up deliberative processes. In this paper, we explore the opportunities and risks associated with applying Large Language Models (LLMs) towards challenges with facilitating, moderating and summarizing the results of Polis engagements. In particular, we demonstrate with pilot experiments using Anthropic's Claude that LLMs can indeed augment human intelligence to help more efficiently run Polis conversations. In particular, we find that summarization capabilities enable categorically new methods with immense promise to empower the public in collective meaning-making exercises. And notably, LLM context limitations have a significant impact on insight and quality of these results. However, these opportunities come with risks. We discuss some of these risks, as well as principles and techniques for characterizing and mitigating them, and the implications for other deliberative or political systems that may employ LLMs. Finally, we conclude with several open future research directions for augmenting tools like Polis with LLMs.