mlsanityData to 14 Oct 2026

Exploring Safety-Utility Trade-Offs in Personalized Language Models

arXiv 2406.11107 ↗First posted 17 Jun 2024cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) become increasingly integrated into daily applications, it is essential to ensure they operate fairly across diverse user demographics. In this work, we show that LLMs suffer from personalization bias, where their performance is impacted when they are personalized to a user's identity. We quantify personalization bias by evaluating the performance of LLMs along two axes - safety and utility. We measure safety by examining how benign LLM responses are to unsafe prompts with and without personalization. We measure utility by evaluating the LLM's performance on various tasks, including general knowledge, mathematical abilities, programming, and reasoning skills. We find that various LLMs, ranging from open-source models like Llama (Touvron et al., 2023) and Mistral (Jiang et al., 2023) to API-based ones like GPT-3.5 and GPT-4o (Ouyang et al., 2022), exhibit significant variance in performance in terms of safety-utility trade-offs depending on the user's identity. Finally, we discuss several strategies to mitigate personalization bias using preference tuning and prompt-based defenses.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 55 of its 75 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

55 of 75 references.

References · 55

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Mistral 7Bcited by 3,164
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  12. 2016Equality of Opportunity in Supervised Learningcited by 2,007
38 more
  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  2. 2022Red Teaming Language Models with Language Modelscited by 1,026
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  5. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  6. 2018A Reductions Approach to Fair Classificationcited by 587
  7. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  8. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  9. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  10. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  11. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  12. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  13. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  14. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  15. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  16. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  17. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  18. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  19. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  20. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  21. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 158
  22. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  23. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 114
  24. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  25. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  26. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  27. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 92
  28. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  29. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 62
  30. 2019Generating Personalized Recipes from Historical User Preferencescited by 53
  31. 2022Kernelized Concept Erasurecited by 48
  32. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  33. 2024Personalized Large Language Modelscited by 44
  34. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  35. 2024Gradient-Based Language Model Red Teamingcited by 34
  36. 2021Uncovering Implicit Gender Bias in Narratives through Commonsense Inferencecited by 32
  37. 2019Towards Controllable and Personalized Review Generationcited by 24
  38. 2018Compact Personalized Models for Neural Machine Translationcited by 19

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 12.

27 papers.

Sort:

By area:

  1. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 71
  2. 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 19
  3. 2025Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactionscited by 12
  4. 2025PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AIcited by 12
  5. 2025The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generationcited by 10
  6. 2025Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Modelscited by 10
  7. 2026One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalizationcited by 9
  8. 2025The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agentscited by 8
  9. 2025The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behaviorcited by 8
  10. 2025The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agentscited by 7
  11. 2025Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answeringcited by 5
  12. 2025Evaluating Large Language Models for Detecting Antisemitismcited by 3
15 more
  1. 2026When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMscited by 3
  2. 2025From Anger to Joy: How Nationality Personas Shape Emotion Attribution in Large Language Modelscited by 2
  3. 2025Mind the Gap! Pathways Towards Unifying AI Safety and Ethics Researchcited by 2
  4. 2026Reflections and New Directions for Human-Centered Large Language Modelscited by 2
  5. 2026OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Setscited by 2
  6. 2026Identity-Robust Language Model Generation via Content Integrity Preservationcited by 1
  7. 2025The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?cited by 0
  8. 2025Debiasing LLMs by Masking Unfairness-Driving Attention Headscited by 0
  9. 2026Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Educationcited by 0
  10. 2026Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users' Perspectives on Opportunities, Risks, and Mitigation Strategiescited by 0
  11. 2026UGID: Unified Graph Isomorphism for Debiasing Large Language Modelscited by 0
  12. 2026Mitigating LLM biases toward spurious social contexts using direct preference optimizationcited by 0
  13. 2026TriAlign: Towards Universal Truth Consistency in Personalized LLM Alignmentcited by 0
  14. 2026SHARD: Safe and Helpful Alignment via Self-Reframing Distillationcited by 0
  15. 2026Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs
  2. 2025Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approach
  3. 2026Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences
  4. 2026Many Preferences, Few Policies: Towards Scalable Language Model Personalization
  5. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
  6. 2025Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
  7. 2026How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants
  8. 2024Mitigating Exaggerated Safety in Large Language Models