mlsanityData to 14 Oct 2026

On the Inevitability of Left-Leaning Political Bias in Aligned Language Models

arXiv 2507.15328 ↗First posted 21 Jul 2025cs.CL cs.CYcited by 4

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

The guiding principle of AI alignment is to train large language models (LLMs) to be harmless, helpful, and honest (HHH). At the same time, there are mounting concerns that LLMs exhibit a left-wing political bias. Yet, the commitment to AI alignment cannot be harmonized with the latter critique. In this article, I argue that intelligent systems that are trained to be harmless and honest must necessarily exhibit left-wing political bias. Normative assumptions underlying alignment objectives inherently concur with progressive moral frameworks and left-wing principles, emphasizing harm avoidance, inclusivity, fairness, and empirical truthfulness. Conversely, right-wing ideologies often conflict with alignment guidelines. Yet, research on political bias in LLMs is consistently framing its insights about left-leaning tendencies as a risk, as problematic, or concerning. This way, researchers are actively arguing against AI alignment, tacitly fostering the violation of HHH principles.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 72 references that are in the map. First posted on arXiv 21 Jul 2025.

Builds on

32 of 72 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024On the Relationship between Truth and Political Bias in Language Models9 Sep
  2. 2024Beyond Preferences in AI Alignment30 Aug
  3. 2024Concept Alignment9 Jan
  4. 2022Methodological reflections for AI alignment research using human feedback22 Dec

Other references · 28

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  4. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  5. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  7. 2020Artificial Intelligence, Values and Alignmentcited by 448
  8. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  9. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  10. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
  11. 2021Alignment of Language Agentscited by 168
  12. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
16 more
  1. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  2. 2024The Political Preferences of LLMscited by 84
  3. 2023Questioning the Survey Responses of Large Language Modelscited by 74
  4. 2024Can multiple-choice questions really be useful in detecting the abilities of LLMs?cited by 69
  5. 2023The Self-Perception and Political Biases of ChatGPTcited by 66
  6. 2024Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMscited by 55
  7. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  8. 2024Assessing Political Bias in Large Language Modelscited by 32
  9. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  10. 2025Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Modelscited by 15
  11. 2024GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancycited by 14
  12. 2025Political Neutrality in AI Is Impossible- But Here Is How to Approximate Itcited by 10
  13. 2024A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutionscited by 8
  14. 2023Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithmscited by 5
  15. 2018Mimetic vs Anchored Value Alignment in Artificial Intelligencecited by 3
  16. 2019Grounding Value Alignment with Ethical Principlescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025What Is The Political Content in LLMs' Pre- and Post-Training Data?cited by 7
  2. 2026M"OVE: A Holistic LLM Benchmark for the German Public Sectorcited by 1
  3. 2026Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Sciencecited by 0
  4. 2026Framing the Narrative: Ideological Mimicry in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025"Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMs
  2. 2025Normative Conflicts and Shallow AI Alignment
  3. 2025Measuring Political Preferences in AI Systems: An Integrative Approach
  4. 2026The Neutral Mask: How Alignment Training Provides Shallow Alignment while Leaving Partisan Structure Intact in a Large Language Model
  5. 2026Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias
  6. 2025Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes
  7. 2025Artificial Authority: From Machine Minds to Political Alignments. An Experimental Analysis of Democratic and Autocratic Biases in Large-Language Models
  8. 2025PReSS: An Automated Black-Box Framework for Evaluating Political Stance Stability in LLMs