# Which Demographics do LLMs Default to During Annotation?

arXiv: https://arxiv.org/abs/2410.08820 · HTML: https://mlsanity.com/paper/2410.08820/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 77 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting](https://mlsanity.com/paper/2309.07034/) (2023)
- [Reasoning Implicit Sentiment with Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.11255/) (2023)
- [Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs](https://mlsanity.com/paper/2402.17649/) (2024)
- [Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://mlsanity.com/paper/2202.12109/) (2022)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [A Practical Survey on Zero-shot Prompt Design for In-context Learning](https://mlsanity.com/paper/2309.13205/) (2023)
- [Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Prediction](https://mlsanity.com/paper/2206.05238/) (2022)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Can Large Language Models Capture Dissenting Human Voices?](https://mlsanity.com/paper/2305.13788/) (2023)
- [Ruddit: Norms of Offensiveness for English Reddit Comments](https://mlsanity.com/paper/2106.05664/) (2021)
- [Emotion Analysis in NLP: Trends, Gaps and Roadmap for Future Directions](https://mlsanity.com/paper/2403.01222/) (2024)
- [Annotation alignment: Comparing LLM and human annotations of conversational safety](https://mlsanity.com/paper/2406.06369/) (2024)
- [Unifying Data Perspectivism and Personalization: An Application to Social Norms](https://mlsanity.com/paper/2210.14531/) (2022)
- [Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting](https://mlsanity.com/paper/2406.11661/) (2024)
- [PromptNER: Prompt Locating and Typing for Named Entity Recognition](https://mlsanity.com/paper/2305.17104/) (2023)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [From Dissonance to Insights: Dissecting Disagreements in Rationale Construction for Case Outcome Classification](https://mlsanity.com/paper/2310.11878/) (2023)
- [Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora](https://mlsanity.com/paper/2209.06701/) (2022)
- [Emotion Ratings: How Intensity, Annotation Confidence and Agreements are Entangled](https://mlsanity.com/paper/2103.01667/) (2021)
- [PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners](https://mlsanity.com/paper/2205.09229/) (2022)

## Built on by (most cited first, 10 of 10)

- [Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensiveness](https://mlsanity.com/paper/2411.08977/) (2024)
- [Reflect: Transparent Principle-Guided Reasoning for Constitutional Alignment at Scale](https://mlsanity.com/paper/2601.18730/) (2026)
- [Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer](https://mlsanity.com/paper/2606.11018/) (2026)
- [Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation](https://mlsanity.com/paper/2511.04706/) (2025)
- [Structure of Basic Human Values in Russian Social Media](https://mlsanity.com/paper/2603.18822/) (2026)
- [Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks](https://mlsanity.com/paper/2603.21404/) (2026)
- [Are Humans as Brittle as Large Language Models?](https://mlsanity.com/paper/2509.07869/) (2025)
- [PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media](https://mlsanity.com/paper/2601.03860/) (2026)
- [Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement](https://mlsanity.com/paper/2607.10590/) (2026)
- [LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts](https://mlsanity.com/paper/2609.00222/) (2026)

## Abstract (arXiv)

Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find it offensive to read a message addressed to a "bro", but a male teenager might find it appropriate. It is therefore important to acknowledge label variations to not under-represent members of a society. Two research directions developed out of this observation in the context of using large language models (LLM) for data annotations, namely (1) studying biases and inherent knowledge of LLMs and (2) injecting diversity in the output by manipulating the prompt with demographic information. We combine these two strands of research and ask the question to which demographics an LLM resorts to when no demographics is given. To answer this question, we evaluate which attributes of human annotators LLMs inherently mimic. Furthermore, we compare non-demographic conditioned prompts and placebo-conditioned prompts (e.g., "you are an annotator who lives in house number 5") to demographics-conditioned prompts ("You are a 45 year old man and an expert on politeness annotation. How do you rate {instance}"). We study these questions for politeness and offensiveness annotations on the POPQUORN data set, a corpus created in a controlled manner to investigate human label variations based on demographics which has not been used for LLM-based analyses so far. We observe notable influences related to gender, race, and age in demographic prompting, which contrasts with previous studies that found no such effects.