# Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts

arXiv: https://arxiv.org/abs/2608.06123 · HTML: https://mlsanity.com/paper/2608.06123/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Model Organisms for Emergent Misalignment](https://mlsanity.com/paper/2506.11613/) (2025)
- [Measuring Political Bias in Large Language Models: What Is Said and How It Is Said](https://mlsanity.com/paper/2403.18932/) (2024)
- [Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters](https://mlsanity.com/paper/2410.24190/) (2024)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [Political-LLM: Large Language Models in Political Science](https://mlsanity.com/paper/2412.06864/) (2024)
- [GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy](https://mlsanity.com/paper/2407.18008/) (2024)
- [Large language models can consistently generate high-quality content for election disinformation operations](https://mlsanity.com/paper/2408.06731/) (2024)
- [Benchmarking LLMs for Political Science: A United Nations Perspective](https://mlsanity.com/paper/2502.14122/) (2025)
- [Mapping Geopolitical Bias in 11 Large Language Models: A Bilingual, Dual-Framing Analysis of U.S.-China Tensions](https://mlsanity.com/paper/2503.23688/) (2025)
- [Large Language Models in Politics and Democracy: A Comprehensive Survey](https://mlsanity.com/paper/2412.04498/) (2024)
- [Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor](https://mlsanity.com/paper/2604.27633/) (2026)
- [Bilingual Bias in Large Language Models: A Taiwan Sovereignty Benchmark Study](https://mlsanity.com/paper/2602.06371/) (2026)
- [Polar: A Benchmark for Evaluating Political Bias in LLMs](https://mlsanity.com/paper/2606.12922/) (2026)

## Abstract (arXiv)

Measuring political bias in large language models (LLMs) remains challenging as it can manifest through subtle differences in framing, argumentation, and legal reasoning that are difficult to capture with a single metric. In this work, we introduce Poli-Bias, a counterfactual framework for measuring whether LLMs treat legally equivalent conflict scenarios differently depending on the countries involved. Poli-Bias compares responses to paired prompts in which country identities are systematically swapped across diverse geopolitical relationships, legal violations, and reasoning tasks. Rather than reducing bias to a single judgment, our framework decomposes response disparities into five interpretable dimensions, revealing how and where unequal treatment manifests. Across 13 contemporary LLMs spanning diverse model families and sizes, we find that country identities and user affiliations can systematically affect how equivalent actions are described, evaluated, and defended under international law. Our results thus establish Poli-Bias as a fine-grained framework for auditing political even-handedness and sycophancy in LLMs.