# PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives arXiv: https://arxiv.org/abs/2505.19558 ยท HTML: https://mlsanity.com/paper/2505.19558/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 33 of its 73 references that are in the map. - First posted on arXiv 26 May 2025. ## Builds on (most cited first, 30 of 33) - [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023) - [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023) - [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017) - [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022) - [BillSum: A Corpus for Automatic Summarization of US Legislation](https://mlsanity.com/paper/1910.00523/) (2019) - [How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis](https://mlsanity.com/paper/2402.05863/) (2024) - [Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2309.17234/) (2023) - [How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments](https://mlsanity.com/paper/2403.11807/) (2024) - [Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method](https://mlsanity.com/paper/2402.15813/) (2024) - [X-Stance: A Multilingual Multi-Target Dataset for Stance Detection](https://mlsanity.com/paper/2003.08385/) (2020) - [Generative Social Choice](https://mlsanity.com/paper/2309.01291/) (2023) - [Opportunities and Risks of LLMs for Scalable Deliberation with Polis](https://mlsanity.com/paper/2306.11932/) (2023) - [No Press Diplomacy: Modeling Multi-Agent Gameplay](https://mlsanity.com/paper/1909.02128/) (2019) - [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024) - [A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context](https://mlsanity.com/paper/1907.03399/) (2019) - [Linear Representations of Political Perspective Emerge in Large Language Models](https://mlsanity.com/paper/2503.02080/) (2025) - [Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMs](https://mlsanity.com/paper/2502.19148/) (2025) - [DarkBench: Benchmarking Dark Patterns in Large Language Models](https://mlsanity.com/paper/2503.10728/) (2025) - [Political-LLM: Large Language Models in Political Science](https://mlsanity.com/paper/2412.06864/) (2024) - [How Susceptible are Large Language Models to Ideological Manipulation?](https://mlsanity.com/paper/2402.11725/) (2024) - [Democratic Policy Development using Collective Dialogues and AI](https://mlsanity.com/paper/2311.02242/) (2023) - [Language Agents as Digital Representatives in Collective Decision-Making](https://mlsanity.com/paper/2502.09369/) (2025) - [Llama meets EU: Investigating the European Political Spectrum through the Lens of LLMs](https://mlsanity.com/paper/2403.13592/) (2024) - [Modelling Political Coalition Negotiations Using LLM-based Agents](https://mlsanity.com/paper/2402.11712/) (2024) - [Benchmarking LLMs for Political Science: A United Nations Perspective](https://mlsanity.com/paper/2502.14122/) (2025) - [Maintaining Common Ground in Dynamic Environments](https://mlsanity.com/paper/2105.14207/) (2021) - [Conversational Grounding: Annotation and Analysis of Grounding Acts and Grounding Units](https://mlsanity.com/paper/2403.16609/) (2024) - [Simulating The U.S. Senate: An LLM-Driven Agent Approach to Modeling Legislative Behavior and Bipartisanship](https://mlsanity.com/paper/2406.18702/) (2024) ## Built on by (most cited first, 2 of 2) - [Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment](https://mlsanity.com/paper/2510.13387/) (2025) - [Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents](https://mlsanity.com/paper/2607.15095/) (2026) ## Abstract (arXiv) Achieving political consensus is crucial yet challenging for the effective functioning of social governance. However, although frontier AI systems represented by large language models (LLMs) have developed rapidly in recent years, their capabilities in this scope are still understudied. In this paper, we introduce PoliCon, a novel benchmark constructed from 2,225 high-quality deliberation records of the European Parliament over 13 years, ranging from 2009 to 2022, to evaluate the ability of LLMs to draft consensus resolutions based on divergent party positions under varying collective decision-making contexts and political requirements. Specifically, PoliCon incorporates four factors to build each task environment for finding different political consensus: specific political issues, political goals, participating parties, and power structures based on seat distribution. We also developed an evaluation framework based on social choice theory for PoliCon, which simulates the real voting outcomes of different political parties to assess whether LLM-generated resolutions meet the requirements of the predetermined political consensus. Our experimental results demonstrate that even state-of-the-art models remain undersatisfied with complex tasks like passing resolutions by a two-thirds majority and addressing security issues, while uncovering their inherent partisan biases and revealing some behaviors LLMs show to achieve the consensus, such as prioritizing the stance of the dominant party instead of uniting smaller parties, which highlights PoliCon's promise as an effective platform for studying LLMs' ability to promote political consensus. The code and dataset are released at https://zowiezhang.github.io/projects/PoliCon.