# LLM Voting: Human Choices and AI Collective Decision Making

arXiv: https://arxiv.org/abs/2402.01766 · HTML: https://mlsanity.com/paper/2402.01766/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 73 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 24 of 24)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023)
- [Mindstorms in Natural Language-Based Societies of Mind](https://mlsanity.com/paper/2305.17066/) (2023)
- [Training Socially Aligned Language Models on Simulated Social Interactions](https://mlsanity.com/paper/2305.16960/) (2023)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [ChatLLM Network: More brains, More intelligence](https://mlsanity.com/paper/2304.12998/) (2023)
- [The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents](https://mlsanity.com/paper/2311.09665/) (2023)
- [Accelerating science with human-aware artificial intelligence](https://mlsanity.com/paper/2306.01495/) (2023)
- [Properties of Multiwinner Voting Rules](https://mlsanity.com/paper/1506.02891/) (2015)
- [Axiomatic Characterization of Committee Scoring Rules](https://mlsanity.com/paper/1604.01529/) (2016)
- [A Polynomial-time Algorithm to Achieve Extended Justified Representation](https://mlsanity.com/paper/1703.10415/) (2017)

## Built on by (most cited first, 30 of 48)

- [Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents](https://mlsanity.com/paper/2405.10467/) (2024)
- [Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents](https://mlsanity.com/paper/2502.11355/) (2025)
- [Multi-Agent Large Language Models for Conversational Task-Solving](https://mlsanity.com/paper/2410.22932/) (2024)
- [Stay Focused: Problem Drift in Multi-Agent Debate](https://mlsanity.com/paper/2502.19559/) (2025)
- [Multi-Agent Debate for LLM Judges with Adaptive Stability Detection](https://mlsanity.com/paper/2510.12697/) (2025)
- [LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra](https://mlsanity.com/paper/2507.15815/) (2025)
- [KVComm: Enabling Efficient LLM Communication through Selective KV Sharing](https://mlsanity.com/paper/2510.03346/) (2025)
- [An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainability](https://mlsanity.com/paper/2504.04462/) (2025)
- [MALLM: Multi-Agent Large Language Models Framework](https://mlsanity.com/paper/2509.11656/) (2025)
- [Dataset Distillation via Committee Voting](https://mlsanity.com/paper/2501.07575/) (2025)
- [MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs](https://mlsanity.com/paper/2505.20740/) (2025)
- [Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendations](https://mlsanity.com/paper/2510.02352/) (2025)
- [Political Actor Agent: Simulating Legislative System for Roll Call Votes Prediction with Large Language Models](https://mlsanity.com/paper/2412.07144/) (2024)
- [Policy Induction: Predicting Startup Success via Explainable Memory-Augmented In-Context Learning](https://mlsanity.com/paper/2505.21427/) (2025)
- [Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models](https://mlsanity.com/paper/2506.14092/) (2025)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [Gender Dynamics and Homophily in a Social Network of LLM Agents](https://mlsanity.com/paper/2602.02606/) (2026)
- [DeFine: Decision-Making with Analogical Reasoning over Factor Profiles](https://mlsanity.com/paper/2410.01772/) (2024)
- [Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making](https://mlsanity.com/paper/2503.05724/) (2025)
- [Adaptive political surveys and GPT-4: Tackling the cold start problem with simulated user interactions](https://mlsanity.com/paper/2503.09311/) (2025)
- [AI-Generated Compromises for Coalition Formation](https://mlsanity.com/paper/2506.06837/) (2025)
- [Addressing the alignment problem in transportation policy making: an LLM approach](https://mlsanity.com/paper/2510.13139/) (2025)
- [ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiation](https://mlsanity.com/paper/2510.25224/) (2025)
- [Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcare](https://mlsanity.com/paper/2512.03097/) (2025)
- [A closer look at how large language models trust humans: patterns and biases](https://mlsanity.com/paper/2504.15801/) (2025)
- [Matching Markets Meet LLMs: Algorithmic Reasoning with Ranked Preferences](https://mlsanity.com/paper/2506.04478/) (2025)
- [A Game-Theoretic Negotiation Framework for Cross-Cultural Consensus in LLMs](https://mlsanity.com/paper/2506.13245/) (2025)
- [E3RG: Building Explicit Emotion-driven Empathetic Response Generation System with Multimodal Large Language Model](https://mlsanity.com/paper/2508.12854/) (2025)
- [Ripple Effect Protocol: Coordinating Agent Populations](https://mlsanity.com/paper/2510.16572/) (2025)
- [Belief Engine: Configurable and Inspectable Stance Dynamics in Multi-Agent LLM Deliberation](https://mlsanity.com/paper/2605.15343/) (2026)

## Abstract (arXiv)

This paper investigates the voting behaviors of Large Language Models (LLMs), specifically GPT-4 and LLaMA-2, their biases, and how they align with human voting patterns. Our methodology involved using a dataset from a human voting experiment to establish a baseline for human preferences and conducting a corresponding experiment with LLM agents. We observed that the choice of voting methods and the presentation order influenced LLM voting outcomes. We found that varying the persona can reduce some of these biases and enhance alignment with human choices. While the Chain-of-Thought approach did not improve prediction accuracy, it has potential for AI explainability in the voting process. We also identified a trade-off between preference diversity and alignment accuracy in LLMs, influenced by different temperature settings. Our findings indicate that LLMs may lead to less diverse collective outcomes and biased assumptions when used in voting scenarios, emphasizing the need for cautious integration of LLMs into democratic processes.