# Polar: A Benchmark for Evaluating Political Bias in LLMs

arXiv: https://arxiv.org/abs/2606.12922 · HTML: https://mlsanity.com/paper/2606.12922/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 80 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 14 of 14)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Lessons from the Trenches on Reproducible Evaluation of Language Models](https://mlsanity.com/paper/2405.14782/) (2024)
- [Large Language Models for Education: A Survey](https://mlsanity.com/paper/2405.13001/) (2024)
- [EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes](https://mlsanity.com/paper/2507.11407/) (2025)
- [Kanana: Compute-efficient Bilingual Language Models](https://mlsanity.com/paper/2502.18934/) (2025)
- [Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameters](https://mlsanity.com/paper/2505.04393/) (2025)
- [Uncovering Political Bias in Large Language Models using Parliamentary Voting Records](https://mlsanity.com/paper/2601.08785/) (2026)
- [HyperCLOVA X THINK Technical Report](https://mlsanity.com/paper/2506.22403/) (2025)
- [Mi:dm 2.0 Korea-centric Bilingual Language Models](https://mlsanity.com/paper/2601.09066/) (2026)
- [No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models](https://mlsanity.com/paper/2503.11985/) (2025)

## Built on by (most cited first, 1 of 1)

- [Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts](https://mlsanity.com/paper/2608.06123/) (2026)

## Abstract (arXiv)

Political bias in large language models (LLMs) is increasingly significant, but difficult to measure reproducibly across political and linguistic contexts. We introduce Polar, a 4,026-instance multiple-choice benchmark that measures political bias through option-level likelihoods rather than prompt-based generation. Polar covers two ideological axes and eight issue categories derived from the Manifesto Project, and evaluates models in parallel across U.S. and South Korean political contexts. Across 38 LLMs, measured bias varies systematically with political context, issue category, model group, and presentation language. All models lean left-progressive on U.S. political content, but show more centered and mixed patterns on South Korean content. Translation experiments further show that presentation language alone can shift measured bias. These findings highlight the need for multilingual and cross-contextual evaluation of political bias in LLMs.