mlsanityData to 7 Oct 2026

Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detection

arXiv 2510.19331 ↗First posted 22 Oct 2025cs.CL cs.CYcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate how personalising Large Language Models (Persona-LLMs) with annotator personas affects their sensitivity to hate speech, particularly regarding biases linked to shared or differing identities between annotators and targets. To this end, we employ Google's Gemini and OpenAI's GPT-4.1-mini models and two persona-prompting methods: shallow persona prompting and a deeply contextualised persona development based on Retrieval-Augmented Generation (RAG) to incorporate richer persona profiles. We analyse the impact of using in-group and out-group annotator personas on the models' detection performance and fairness across diverse social groups. This work bridges psychological insights on group identity with advanced NLP techniques, demonstrating that incorporating socio-demographic attributes into LLMs can address bias in automated hate speech detection. Our results highlight both the potential and limitations of persona-based approaches in reducing bias, offering valuable insights for developing more equitable hate speech detection systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

18 of 38 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  4. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  5. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  6. 2024Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targetscited by 31
  7. 2024Large Language Models Show Human-like Social Desirability Biases in Survey Responsescited by 27
  8. 2022How Hate Speech Varies by Target Identity: A Computational Analysiscited by 23
  9. 2024Hate Personified: Investigating the role of LLMs in content moderationcited by 23
  10. 2023Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Modelscited by 21
  11. 2024Personas with Attitudes: Controlling LLMs for Diverse Data Annotationcited by 21
  12. 2024Contextualized Counterspeech: Strategies for Adaptation, Personalization, and Evaluationcited by 15
6 more
  1. 2024"Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definitioncited by 7
  2. 2025Evaluating how LLM annotations represent diverse views on contentious topicscited by 6
  3. 2025The Impact of Persona-based Political Perspectives on Hateful Content Detectioncited by 5
  4. 2022BERT-based Ensemble Approaches for Hate Speech Detectioncited by 4
  5. 2025Leveraging a Multi-Agent LLM-Based System to Educate Teachers in Hate Incidents Managementcited by 3
  6. 2024Towards Identity-Aware Cross-Modal Retrieval: a Dataset and a Baselinecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Improving Implicit Hate Speech Detection via a Community-Driven Multi-Agent Frameworkcited by 0
  2. 2026Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Persona Prompting as a Lens on LLM Social Reasoning
  2. 2025Confident, Calibrated, or Complicit: Safety Alignment and Ideological Bias in LLM Hate Speech Detection
  3. 2026Attribute-Based Diagnosis of LLM Alignment with Hate Speech Annotations
  4. 2025Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate
  5. 2025Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?
  6. 2025Us-vs-Them bias in Large Language Models ↗
  7. 2025HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns
  8. 2025MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanation