# Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Level

arXiv: https://arxiv.org/abs/2608.14692 · HTML: https://mlsanity.com/paper/2608.14692/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 108 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 30 of 42)

- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers](https://mlsanity.com/paper/2311.09180/) (2023)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [A Complete Survey on LLM-based AI Chatbots](https://mlsanity.com/paper/2406.16937/) (2024)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances](https://mlsanity.com/paper/2409.11360/) (2024)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval](https://mlsanity.com/paper/2407.20371/) (2024)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Personalized Generation In Large Model Era: A Survey](https://mlsanity.com/paper/2503.02614/) (2025)
- [AI auditing: The Broken Bus on the Road to AI Accountability](https://mlsanity.com/paper/2401.14462/) (2024)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [User Modeling and User Profiling: A Comprehensive Survey](https://mlsanity.com/paper/2402.09660/) (2024)
- [Accumulating Context Changes the Beliefs of Language Models](https://mlsanity.com/paper/2511.01805/) (2025)
- [Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness](https://mlsanity.com/paper/2303.17555/) (2023)
- [Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach](https://mlsanity.com/paper/2407.14779/) (2024)
- [CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions](https://mlsanity.com/paper/2508.01674/) (2025)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [The Case for "Thick Evaluations" of Cultural Representation in AI](https://mlsanity.com/paper/2503.19075/) (2025)
- [WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI](https://mlsanity.com/paper/2501.01397/) (2025)

## Abstract (arXiv)

Personalized, generative AI systems increasingly adapt their behavior to individual users over time, fundamentally changing model behavior. While existing auditing approaches have been effective at surfacing harms in non-personalized contexts, they often rely on static, simulated evaluations and definitions of harm that aggregate across broad, group categories. In this position paper, we argue that such approaches can fail to capture emergent harms in personalized generative AI systems, where harms surface through interpretations of ongoing interaction and evolve with user history. We identify three presuppositions underlying many harm auditing paradigms: that harms can be (1) specified outside real-world interaction, (2) defined non-pluralistically within groups, and (3) treated as static. One might argue that personalized systems could simply learn definitions of what constitutes harm to individual users through repeated interactions. However, we argue that attempts to surface user harms through deeper personalization risk imposing asymmetric burdens of labor and privacy on marginalized users. Consequently, we propose reframing understandings of harm as adaptive, user- and community-centered processes, and outline design directions that shift auditing from retrospective evaluation toward infrastructures that support ongoing articulation of harm in interaction. Our work highlights the need for auditing and design practices that better reflect the pluralistic and evolving nature of harm understanding in personalized generative AI systems.