mlsanityData to 7 Oct 2026

Token-Level Entropy Reveals Demographic Disparities in Large Language Models

arXiv 2501.19337 ↗First posted 31 Jan 2025cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

A name alone measurably reshapes a language model's next-token distribution before a single token is sampled. We measure full-vocabulary Shannon entropy of the next-token distribution across six open-weight model families on 5,760 sentence-completion prompts in which race and gender are signaled only by a first name. Black-associated names co-occur with higher first-token entropy and more diverse continuations than White-associated names -- directionally consistent in all six instruction-tuned models under shared raw-text input, all six base checkpoints, and, for output diversity, five of six models under native chat formatting -- opposite to the homogeneity bias documented under explicit group labels (Lee et al., 2024). The gap persists under tokenization and frequency controls and on a frequency-matched name subset; per-prompt effects are small (d = 0.06-0.16) but uniformly signed (template-level paired d = 0.66-1.08). Gender points the other way, additively with race. First-token entropy attenuates sharply under chat-formatted input, and explicit group-label probing is mostly null or reversed; a variance-matched comparison locates the output-diversity disparity in heterogeneity across name-conditioned continuations -- a dimension a fixed group label cannot express. Probing methodology shapes not only whether a disparity is detected but which direction it takes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 77 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

15 of 77 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  6. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 165
  7. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  8. 2022On Decoding Strategies for Neural Text Generatorscited by 89
  9. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 89
  10. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  11. 2024What's in a Name? Auditing Large Language Models for Race and Gender Biascited by 50
  12. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 39
3 more
  1. 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
  2. 2023Should We Attend More or Less? Modulating Attention for Fairnesscited by 11
  3. 2024Vision-Language Models Generate More Homogeneous Stories for Phenotypically Black Individualscited by 6

Built on by

Not cited yet.