mlsanityData to 14 Oct 2026

A Multifaceted Analysis of Social Biases in Large Language Models

arXiv 2512.15792 ↗First posted 16 Dec 2025cs.CY cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making. However, ensuring that these models uphold fairness across varied contexts is critical to their safe and responsible deployment. In this study, we undertake a comprehensive examination of four widely adopted LLMs, probing their underlying biases and inclinations across the dimensions of politics, ideology, alliance, language, and gender. Through a series of carefully designed experiments, we investigate their political neutrality using news summarization, ideological biases through news stance classification, tendencies toward specific geopolitical alliances via United Nations voting patterns, language bias in the context of multilingual story completion, and gender-related affinities as revealed by responses to the World Values Survey. Results indicate that while the LLMs are aligned to be neutral and impartial, they still show biases and affinities of different types.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 16 Dec 2025.

Builds on

33 of 55 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  4. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  5. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  6. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  7. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  8. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  9. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  10. 2023Generative Language Models Exhibit Social Identity Biasescited by 111
  11. 2020We Can Detect Your Bias: Predicting the Political Ideology of News Articlescited by 92
  12. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
21 more
  1. 2024The Political Preferences of LLMscited by 84
  2. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  3. 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Biascited by 53
  4. 2023ROBBIE: Robust Bias Evaluation of Large Generative Language Modelscited by 53
  5. 2023Bias of AI-Generated Content: An Examination of News Produced by Large Language Modelscited by 38
  6. 2020Analyzing Political Bias and Unfairness in News Articles at Different Levels of Granularitycited by 32
  7. 2021Unpacking the Interdependent Systems of Discrimination: Ableist Bias in NLP Systems through an Intersectional Lenscited by 30
  8. 2024AI-AI Bias: large language models favor communications generated by large language modelscited by 26
  9. 2024Bias Unveiled: Investigating Social Bias in LLM-Generated Codecited by 25
  10. 2023An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Modelscited by 24
  11. 2023NBIAS: A Natural Language Processing Framework for Bias Identification in Textcited by 23
  12. 2022Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruningcited by 21
  13. 2023Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistancecited by 15
  14. 2024IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicatorscited by 15
  15. 2024Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?cited by 8
  16. 2024Gender Bias in Text-to-Video Generation Models: A case study of Soracited by 8
  17. 2025No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomycited by 8
  18. 2021Reinforcement Learning with Feedback from Multiple Humans with Diverse Skillscited by 6
  19. 2022Power of Explanations: Towards automatic debiasing in hate speech detectioncited by 5
  20. 2024I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Modelscited by 5
  21. 2025Debiasing CLIP: Interpreting and Correcting Bias in Attention Headscited by 4

Built on by

Not cited yet.