mlsanityData to 14 Oct 2026

When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries

arXiv 2410.09978 ↗First posted 13 Oct 2024cs.CL cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

In an era where societal narratives are increasingly shaped by algorithmic curation, investigating the political neutrality of LLMs is an important research question. This study presents a fresh perspective on quantifying the political neutrality of LLMs through the lens of abstractive text summarization of polarizing news articles. We consider five pressing issues in current US politics: abortion, gun control/rights, healthcare, immigration, and LGBTQ+ rights. Via a substantial corpus of 20,344 news articles, our study reveals a consistent trend towards pro-Democratic biases in several well-known LLMs, with gun control and healthcare exhibiting the most pronounced biases (max polarization differences of -9.49% and -6.14%, respectively). Further analysis uncovers a strong convergence in the vocabulary of the LLM outputs for these divisive topics (55% overlap for Democrat-leaning representations, 52% for Republican). Being months away from a US election of consequence, we consider our findings important.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 20 of its 59 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

20 of 59 references.

References · 20

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Mistral 7Bcited by 3,164
  3. 2020Learning to summarize from human feedbackcited by 2,588
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  5. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  6. 2023PaLM 2 Technical Reportcited by 1,162
  7. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  9. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  10. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 260
  11. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
  12. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
8 more
  1. 2019Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootingscited by 67
  2. 2022POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detectioncited by 51
  3. 2020A Framework for the Computational Linguistic Analysis of Dehumanizationcited by 31
  4. 2020We Don't Speak the Same Language: Interpreting Polarization through Machine Translationcited by 23
  5. 2022KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Mediacited by 20
  6. 2023Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensivecited by 16
  7. 2023Monoculture in Matching Marketscited by 5
  8. 2024The Silent Curriculum: How Does LLM Monoculture Shape Educational Content and Its Accessibility?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMscited by 6
  2. 2026When Bigger Isn't Better: A Comprehensive Fairness Evaluation of Political Bias in Multi-News Summarisationcited by 2
  3. 2025Input Order Shapes LLM Semantic Alignment in Multi-Document Summarizationcited by 1
  4. 2026DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysiscited by 1
  5. 2025Benchmarking Gender and Political Bias in Large Language Modelscited by 0
  6. 2025The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Modelscited by 0
  7. 2026Framing the Narrative: Ideological Mimicry in Large Language Modelscited by 0