mlsanityData to 7 Oct 2026

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

arXiv 2305.08283 ↗First posted 15 May 2023cs.CLcited by 257

Not on the map. Why?

Cited by 20 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language models (LMs) are pretrained on diverse data sources, including news, discussion forums, books, and online encyclopedias. A significant portion of this data includes opinions and perspectives which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. Our work develops new methods to (1) measure political biases in LMs trained on such corpora, along social and economic axes, and (2) measure the fairness of downstream NLP models trained on top of politically biased LMs. We focus on hate speech and misinformation detection, aiming to empirically quantify the effects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks. Our findings reveal that pretrained LMs do have political leanings that reinforce the polarization present in pretraining corpora, propagating social biases into hate speech predictions and misinformation detectors. We discuss the implications of our findings for NLP research and propose future directions to mitigate unfairness.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). Builds on 65 of its 134 references that are in the map. First posted on arXiv 15 May 2023.

Builds on

65 of 134 references.

References · 65

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
38 more
  1. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  2. 2019Language Models as Knowledge Bases?cited by 1,922
  3. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  5. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  10. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  12. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  13. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  14. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  15. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  16. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  17. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  18. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
  19. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  20. 2020The Pushshift Reddit Datasetcited by 289
  21. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  22. 2019Measuring Bias in Contextualized Word Representationscited by 275
  23. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  24. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  25. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  26. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  27. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  28. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  29. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  30. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  31. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  32. 201850 Years of Test (Un)fairness: Lessons for Machine Learningcited by 101
  33. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  34. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  35. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  36. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  37. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  38. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 34; 2024: 103; 2025: 78; 2026: 42.

257 papers.

Sort:

By area:

  1. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  2. 2023Cognitive Architectures for Language Agentscited by 422
  3. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  4. 2023Challenges and Applications of Large Language Modelscited by 251
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  6. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  7. 2024A Roadmap to Pluralistic Alignmentcited by 175
  8. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  9. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  10. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  11. 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
  12. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
38 more
  1. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  2. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  3. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  4. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  5. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  6. 2024The Political Preferences of LLMscited by 83
  7. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  8. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  9. 2023Questioning the Survey Responses of Large Language Modelscited by 70
  10. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 64
  11. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  12. 2023Large Language Models to Identify Social Determinants of Health in Electronic Health Recordscited by 55
  13. 2024Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMscited by 55
  14. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 54
  15. 2023On the steerability of large language models toward data-driven personascited by 53
  16. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  17. 2023You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instrumentscited by 48
  18. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 48
  19. 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 47
  20. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  21. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
  22. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  23. 2024Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networkscited by 39
  24. 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
  25. 2024Data-driven Discovery with Large Generative Modelscited by 33
  26. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 33
  27. 2023This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Modelscited by 31
  28. 2024On the Relationship between Truth and Political Bias in Language Modelscited by 31
  29. 2025Linear Representations of Political Perspective Emerge in Large Language Modelscited by 30
  30. 2024AI for social science and social science of AI: A Surveycited by 29
  31. 2024LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Educationcited by 28
  32. 2024Empirical evidence of Large Language Model's influence on human spoken communicationcited by 27
  33. 2024Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligencecited by 27
  34. 2024How Susceptible are Large Language Models to Ideological Manipulation?cited by 26
  35. 2024Don't Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detectioncited by 25
  36. 2024Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Modelcited by 25
  37. 2025When One LLM Drools, Multi-LLM Collaboration Rulescited by 25
  38. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 24

50 most cited of 257 All 257 (JSON)