mlsanityData to 14 Oct 2026

Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement

arXiv 2607.10590 ↗First posted 12 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We investigate how annotator demographic attributes, supplied as prompt cues, shape the alignment between large language model (LLM) predictions and human annotations across five tasks. Using five open-source LLMs, we systematically vary the number and composition of demographic components in the prompt, spanning every combination from single-attribute through full-attribute configurations. Our experiments reveal three principal findings. First, alignment consistently peaks with one to three high-signal attributes and degrades under the full attribute set, establishing a clear over-specification threshold. Second, the overall magnitude of demographic influence on human annotations does not predict which attributes improve LLM alignment; instead, both the learnability and the directional coherence of each attribute's annotation signal need to be considered jointly. Third, neuron probing reveals that specialized activation correlates with alignment gains only under coherent annotation signals, and that activation volume alone does not imply steerability. Together, these results demonstrate that demographic prompting is not a monolithic intervention: its utility is highly context-dependent, shaped by attribute signal quality, task characteristics, and model architecture.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 12 Jul 2026.

Builds on

23 of 38 references.

References · 23

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  6. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
  7. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
  8. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  9. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 81
  10. 2021Designing Toxic Content Classification for a Diversity of Perspectivescited by 65
  11. 2023The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographicscited by 46
  12. 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 40
11 more
  1. 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
  2. 2024Intrinsic Test of Unlearning Using Parametric Knowledge Tracescited by 29
  3. 2021Ruddit: Norms of Offensiveness for English Reddit Commentscited by 27
  4. 2025Disentangling Language and Culture for Evaluating Multilingual Large Language Modelscited by 16
  5. 2025Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editingcited by 14
  6. 2025Order Matters: Investigate the Position Bias in Multi-constraint Instruction Followingcited by 12
  7. 2025Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metriccited by 11
  8. 2024Which Demographics do LLMs Default to During Annotation?cited by 10
  9. 2022Aligned with Whom? Direct and social goals for AI systemscited by 7
  10. 2024Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensivenesscited by 7
  11. 2025Evaluating how LLM annotations represent diverse views on contentious topicscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
  2. 2024Evaluating Human Alignment and Model Faithfulness of LLM Rationale
  3. 2025Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models
  4. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignment
  5. 2023Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
  6. 2026On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
  7. 2026Persona Prompting as a Lens on LLM Social Reasoning
  8. 2023Psychometric Predictive Power of Large Language Models