mlsanityData to 14 Oct 2026

Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings

arXiv 2411.16527 ↗First posted 25 Nov 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are the foundation of the current successes of artificial intelligence (AI), however, they are unavoidably biased. To effectively communicate the risks and encourage mitigation efforts these models need adequate and intuitive descriptions of their discriminatory properties, appropriate for all audiences of AI. We suggest bias profiles with respect to stereotype dimensions based on dictionaries from social psychology research. Along these dimensions we investigate gender bias in contextual embeddings, across contexts and layers, and generate stereotype profiles for twelve different LLMs, demonstrating their intuition and use case for exposing and visualizing bias.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

22 of 42 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  10. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  11. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  12. 2019On Measuring Social Biases in Sentence Encoderscited by 398
10 more
  1. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  2. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 135
  3. 2018SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentimentcited by 26
  4. 2022A Robust Bias Mitigation Procedure Based on the Stereotype Content Modelcited by 24
  5. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  6. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  7. 2020FrameAxis: Characterizing Microframe Bias and Intensity with Word Embeddingcited by 21
  8. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  9. 2020The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddingscited by 15
  10. 2023SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddingscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025MIST: Towards Multi-dimensional Implicit BiaS Evaluation of LLMs for Theory of Mindcited by 1
  2. 2026STEREODISCO: Discovering Stereotypicality in LLMscited by 1
  3. 2026Cross-Cultural Value Attribution in Large Vision-Language Modelscited by 0