mlsanityData to 7 Oct 2026

Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor

arXiv 2604.27633 ↗First posted 30 Apr 2026cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires, which typically place frontier models on the political left. A parallel literature shows that LLMs are sycophantic: they adapt their answers to the views, identities, and expectations of the user. We show that these findings are linked: standard political-bias audits partly capture sycophantic accommodation to the inferred auditor. We employ a factorial experiment across three major audit instruments--the Political Compass Test, the Pew Political Typology, and 1,540 partisan-benchmarked Pew American Trends Panel items--administered to six frontier LLMs while varying only the asker's stated identity (N = 30,990 responses). At baseline, all six models lean left. When the asker identifies as a conservative Republican, responses shift sharply: the share of items closer to Democrats falls by 28-62 percentage points, and all six models move right of center. A mirror-image progressive-Democrat cue produces little change; rightward accommodation is 8.0× larger than leftward. When asked who the default asker is, models identify an auditor, researcher, or academic; when asked what answer that asker expects, they select the Democrat-coded option 75% of the time, nearly the rate under an explicit progressive cue. These patterns are inconsistent with a purely fixed model ideology and indicate that single-prompt audits capture an interaction between model and inferred interlocutor. Political bias in LLMs is therefore not a fixed point on an ideological scale but a response profile that must be mapped across realistic interlocutors.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 12 of its 27 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

12 of 27 references.

References · 12

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  5. 2023Whose Opinions Do Language Models Reflect?cited by 759
  6. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  7. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  8. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
  9. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  10. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  11. 2024Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMscited by 55
  12. 2023Aligning Large Language Models with Human Opinions through Persona Selection and Value--Belief--Norm Reasoningcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.