# Personas with Attitudes: Controlling LLMs for Diverse Data Annotation

arXiv: https://arxiv.org/abs/2410.11745 · HTML: https://mlsanity.com/paper/2410.11745/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting](https://mlsanity.com/paper/2309.07034/) (2023)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics](https://mlsanity.com/paper/2306.11559/) (2023)
- ["HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media](https://mlsanity.com/paper/2304.10619/) (2023)
- [Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs](https://mlsanity.com/paper/2311.09730/) (2023)
- [The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation](https://mlsanity.com/paper/2405.01299/) (2024)
- [Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation](https://mlsanity.com/paper/2307.12973/) (2023)
- [Artificial Data, Real Insights: Evaluating Opportunities and Risks of Expanding the Data Ecosystem with Synthetic Data](https://mlsanity.com/paper/2408.15260/) (2024)

## Built on by (most cited first, 21 of 21)

- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions](https://mlsanity.com/paper/2502.20897/) (2025)
- [Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas](https://mlsanity.com/paper/2412.14843/) (2024)
- [BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation](https://mlsanity.com/paper/2502.01697/) (2025)
- [SynthTRIPs: A Knowledge-Grounded Framework for Benchmark Query Generation for Personalized Tourism Recommenders](https://mlsanity.com/paper/2504.09277/) (2025)
- [Doubly-Robust LLM-as-a-Judge: Externally Valid Estimation with Imperfect Personas](https://mlsanity.com/paper/2509.22957/) (2025)
- [The Impact of Persona-based Political Perspectives on Hateful Content Detection](https://mlsanity.com/paper/2502.00385/) (2025)
- [Persona-Based Synthetic Data Generation Using Multi-Stage Conditioning with Large Language Models for Emotion Recognition](https://mlsanity.com/paper/2507.13380/) (2025)
- [Political Ideology Shifts in Large Language Models](https://mlsanity.com/paper/2508.16013/) (2025)
- [German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies](https://mlsanity.com/paper/2511.21722/) (2025)
- [The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study](https://mlsanity.com/paper/2605.20767/) (2026)
- [Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detection](https://mlsanity.com/paper/2510.19331/) (2025)
- [Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer](https://mlsanity.com/paper/2606.11018/) (2026)
- [How Well Do Large Language Models Capture Human Personality?](https://mlsanity.com/paper/2606.18263/) (2026)
- [Ideology-Based LLMs for Content Moderation](https://mlsanity.com/paper/2510.25805/) (2025)
- [Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation](https://mlsanity.com/paper/2608.10385/) (2026)
- [Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups](https://mlsanity.com/paper/2510.06974/) (2025)
- [Misalignment of LLM-Generated Personas with Human Perceptions in Low-Resource Settings](https://mlsanity.com/paper/2512.02058/) (2025)
- [PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media](https://mlsanity.com/paper/2601.03860/) (2026)
- [Chehre: An Emoji-Prompted Dataset to Explore Perceptual Flexibility in Video Language Models](https://mlsanity.com/paper/2606.21657/) (2026)
- [LLM-Ideoplasticity: Measuring Ideological Plasticity in the Political Behavior of LLMs as a Context-Conditioned Distribution](https://mlsanity.com/paper/2606.28335/) (2026)

## Abstract (arXiv)

We present a novel approach for enhancing diversity and control in data annotation tasks by personalizing large language models (LLMs). We investigate the impact of injecting diverse persona descriptions into LLM prompts across two studies, exploring whether personas increase annotation diversity and whether the impacts of individual personas on the resulting annotations are consistent and controllable. Our results show that persona-prompted LLMs produce more diverse annotations than LLMs prompted without personas and that these effects are both controllable and repeatable, making our approach a suitable tool for improving data annotation in subjective NLP tasks like toxicity detection.