Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models
Not on the map. Why?
Abstract (arXiv)
As large language models (LLMs) become increasingly embedded in civic, educational, and political information environments, concerns about their potential political bias have grown. Prior research often evaluates such bias through simulated personas or predefined ideological typologies, which may introduce artificial framing effects or overlook how models behave in general use scenarios. This study adopts a persona-free, topic-specific approach to evaluate political behavior in LLMs, reflecting how users typically interact with these systems-without ideological role-play or conditioning. We introduce a two-dimensional framework: one axis captures partisan orientation on highly polarized topics (e.g., abortion, immigration), and the other assesses sociopolitical engagement on less polarized issues (e.g., climate change, foreign policy). Using survey-style prompts drawn from the ANES and Pew Research Center, we analyze responses from 43 LLMs developed in the U.S., Europe, China, and the Middle East. We propose an entropy-weighted bias score to quantify both the direction and consistency of partisan alignment, and identify four behavioral clusters through engagement profiles. Findings show most models lean center-left or left ideologically and vary in their nonpartisan engagement patterns. Model scale and openness are not strong predictors of behavior, suggesting that alignment strategy and institutional context play a more decisive role in shaping political expression.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 11 of its 44 references that are in the map. First posted on arXiv 21 Dec 2024.
Builds on
11 of 44 references.
References · 11
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
- 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
- 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
- 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
- 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
- 2024Assessing Political Bias in Large Language Modelscited by 32
- 2024Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personascited by 16
- 2023Measurement in the Age of LLMs: An Application to Ideological Scalingcited by 11
Built on by
Citing papers per year: 2025: 5; 2026: 8.
13 papers.
Sort:
By area:
- 2025Emergence of polarization in networks of large language model agentscited by 25
- 2025Benchmarking Overton Pluralism in LLMscited by 7
- 2025Tracing Partisan Bias to Its Emotional Fingerprints: A Computational Approach to Mitigationcited by 0
- 2025Recalibrating the Compass: Integrating Large Language Models into Classical Research Methodscited by 0
- 2025Left Leaning Models: How AI Evaluates Economic Policy?cited by 0
- 2026Hidden Topics: Measuring Sensitive AI Beliefs with List Experimentscited by 0
- 2026Multi-Agent Strategic Games with LLMscited by 0
- 2026NewsLens: A Multi-Agent Framework for Adversarial News Bias Navigationcited by 0
- 2026Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologiescited by 0
- 2026Who Verifies the Benchmark? Decentralizing Trust in Large Language Model Evaluationcited by 0
- 2026Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Studycited by 0
- 2026Position: The Alignment Community is Unintentionally Building a Censor's Toolkitcited by 0
1 more
- 2026Framing the Narrative: Ideological Mimicry in Large Language Models29 Sep 2026
- 2026Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study12 Aug 2026
- 2026Who Verifies the Benchmark? Decentralizing Trust in Large Language Model Evaluation7 Aug 2026
- 2026Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies16 Jul 2026
- 2026Position: The Alignment Community is Unintentionally Building a Censor's Toolkit5 Jun 2026
- 2026NewsLens: A Multi-Agent Framework for Adversarial News Bias Navigation17 May 2026
- 2026Multi-Agent Strategic Games with LLMs5 May 2026
- 2026Hidden Topics: Measuring Sensitive AI Beliefs with List Experiments25 Feb 2026
- 2025Benchmarking Overton Pluralism in LLMs1 Dec 2025
- 2025Left Leaning Models: How AI Evaluates Economic Policy?21 Jul 2025
- 2025Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods26 May 2025
- 2025Emergence of polarization in networks of large language model agents9 Jan 2025
- 2025Emergence of polarization in networks of large language model agentscited by 25
- 2026Multi-Agent Strategic Games with LLMscited by 0
- 2026Multi-Agent Strategic Games with LLMs5 May 2026
- 2025Emergence of polarization in networks of large language model agents9 Jan 2025
- 2025Benchmarking Overton Pluralism in LLMscited by 7
- 2025Tracing Partisan Bias to Its Emotional Fingerprints: A Computational Approach to Mitigationcited by 0
- 2025Recalibrating the Compass: Integrating Large Language Models into Classical Research Methodscited by 0
- 2025Left Leaning Models: How AI Evaluates Economic Policy?cited by 0
- 2026Hidden Topics: Measuring Sensitive AI Beliefs with List Experimentscited by 0
- 2026NewsLens: A Multi-Agent Framework for Adversarial News Bias Navigationcited by 0
- 2026Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologiescited by 0
- 2026Who Verifies the Benchmark? Decentralizing Trust in Large Language Model Evaluationcited by 0
- 2026Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Studycited by 0
- 2026Framing the Narrative: Ideological Mimicry in Large Language Modelscited by 0
- 2026Framing the Narrative: Ideological Mimicry in Large Language Models29 Sep 2026
- 2026Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study12 Aug 2026
- 2026Who Verifies the Benchmark? Decentralizing Trust in Large Language Model Evaluation7 Aug 2026
- 2026Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies16 Jul 2026
- 2026NewsLens: A Multi-Agent Framework for Adversarial News Bias Navigation17 May 2026
- 2026Hidden Topics: Measuring Sensitive AI Beliefs with List Experiments25 Feb 2026
- 2025Benchmarking Overton Pluralism in LLMs1 Dec 2025
- 2025Left Leaning Models: How AI Evaluates Economic Policy?21 Jul 2025
- 2025Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods26 May 2025
- 2025Tracing Partisan Bias to Its Emotional Fingerprints: A Computational Approach to Mitigation2 Jan 2025