mlsanityData to 14 Oct 2026

Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering

arXiv 2607.20487 ↗First posted 2 Jun 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly used to answer questions about political information, including in election-adjacent information settings where factual errors and ideological distortions are high-stakes. We present a reproducible measurement framework that treats hallucinations, unsupported statements in document-grounded QA, as diagnostic signals of ideological drift. Using 21,727 expert-labeled U.S. political news articles from QBias spanning left, center, and right sources, we (i) generate an article-specific question, (ii) elicit document-grounded answers from three open-weight LLMs and one proprietary model, (iii) detect sentence-level hallucinations via reference-based comparison, (iv) classify the ideological valence of hallucinated sentences with a fine-tuned stance classifier, and (v) probe output logits to relate token-level uncertainty to hallucination and drift. Hallucination rates vary substantially across models and concentrate in contentious topics, while source-ideology differences in hallucination frequency are modest. In contrast, hallucination content exhibits robust leftward drift: a majority of hallucinated sentences are classified as left-leaning, including among hallucinations generated from right-leaning sources. Logit-level analysis shows hallucinations arise in high-entropy generation contexts, and in some models uncertainty also predicts leftward drift, consistent with an "uncertainty to guessing" mechanism. We discuss implications for auditing AI-mediated political information and for designing safeguards in election-relevant deployments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

21 of 34 references.

References · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  5. 2021Ethical and social risks of harm from Language Modelscited by 966
  6. 2017Selective Classification for Deep Neural Networkscited by 852
  7. 2023Whose Opinions Do Language Models Reflect?cited by 773
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  10. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  11. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  12. 2025Why Language Models Hallucinatecited by 220
9 more
  1. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  2. 2024Quantifying Variance in Evaluation Benchmarkscited by 54
  3. 2024Assessing Political Bias in Large Language Modelscited by 32
  4. 2024ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Modelscited by 14
  5. 2025What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practicescited by 11
  6. 2025Measuring Political Preferences in AI Systems: An Integrative Approachcited by 9
  7. 2026Uncovering Political Bias in Large Language Models using Parliamentary Voting Recordscited by 8
  8. 2023Qbias -- A Dataset on Media Bias in Search Queries and Query Suggestionscited by 6
  9. 2025Detecting Linguistic Bias in Government Documents Using Large language Modelscited by 1

Built on by

Not cited yet.