# First-Person Fairness in Chatbots

arXiv: https://arxiv.org/abs/2410.19803 · HTML: https://mlsanity.com/paper/2410.19803/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2024.

## Built on by (most cited first, 20 of 20)

- [Clio: Privacy-Preserving Insights into Real-World AI Use](https://mlsanity.com/paper/2412.13678/) (2024)
- [Emergent Coordination in Multi-Agent Language Models](https://mlsanity.com/paper/2510.05174/) (2025)
- [HumT DumT: Measuring and controlling human-like language in LLMs](https://mlsanity.com/paper/2502.13259/) (2025)
- [The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior](https://mlsanity.com/paper/2509.19364/) (2025)
- [On the Mutual Influence of Gender and Occupation in LLM Representations](https://mlsanity.com/paper/2503.06792/) (2025)
- [Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias](https://mlsanity.com/paper/2601.18486/) (2026)
- [Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMs](https://mlsanity.com/paper/2601.23001/) (2026)
- [Conflicts in Texts: Data, Implications and Challenges](https://mlsanity.com/paper/2504.19472/) (2025)
- [Do Biased Models Have Biased Thoughts?](https://mlsanity.com/paper/2508.06671/) (2025)
- [Language Models That Walk the Talk: A Framework for Formal Fairness Certificates](https://mlsanity.com/paper/2505.12767/) (2025)
- [Fairness Evaluation of Large Language Models in Academic Library Reference Services](https://mlsanity.com/paper/2507.04224/) (2025)
- [NeedleChain: Measuring Intact Context Comprehension Capability of Large Language Models](https://mlsanity.com/paper/2507.22411/) (2025)
- [Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrieval](https://mlsanity.com/paper/2510.02938/) (2025)
- [Equitable Evaluation via Elicitation](https://mlsanity.com/paper/2602.21327/) (2026)
- [StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs](https://mlsanity.com/paper/2605.10442/) (2026)
- [Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research](https://mlsanity.com/paper/2606.12247/) (2026)
- [A Roadmap to Impactful Pluralistic Alignment Research](https://mlsanity.com/paper/2607.22305/) (2026)
- [Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models](https://mlsanity.com/paper/2608.29590/) (2026)
- [Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models](https://mlsanity.com/paper/2609.34065/) (2026)
- [Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts](https://mlsanity.com/paper/2610.07798/) (2026)

## Abstract (arXiv)

Evaluating chatbot fairness is crucial given their rapid proliferation, yet typical chatbot tasks (e.g., resume writing, entertainment) diverge from the institutional decision-making tasks (e.g., resume screening) which have traditionally been central to discussion of algorithmic fairness. The open-ended nature and diverse use-cases of chatbots necessitate novel methods for bias assessment. This paper addresses these challenges by introducing a scalable counterfactual approach to evaluate "first-person fairness," meaning fairness toward chatbot users based on demographic characteristics. Our method employs a Language Model as a Research Assistant (LMRA) to yield quantitative measures of harmful stereotypes and qualitative analyses of demographic differences in chatbot responses. We apply this approach to assess biases in six of our language models across millions of interactions, covering sixty-six tasks in nine domains and spanning two genders and four races. Independent human annotations corroborate the LMRA-generated bias evaluations. This study represents the first large-scale fairness evaluation based on real-world chat data. We highlight that post-training reinforcement learning techniques significantly mitigate these biases. This evaluation provides a practical methodology for ongoing bias monitoring and mitigation.