mlsanityData to 14 Oct 2026

When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

arXiv 2609.22102 ↗First posted 12 Aug 2026cs.SE cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts the technical quality of generated code. We conduct a large-scale empirical study of persona-induced bias in LLM-based code generation, focusing a proprietary model (Gemini 2.5 Pro) and an open-weight model (GPT-OSS-120B). Using 18 demographic personas spanning nationality, gender, and experience level, we compare persona-induced prompts against a neutral baseline. Across 35,000+ generated programs, we analyze demographic marker leakage in reasoning and responses, as well as differences in functional correctness, maintainability, code style, and security. Our results show that demographic cues are frequently reflected in LLM reasoning and outputs. Demographic markers appear in up to 65% of responses and 70% of reasoning traces, despite being semantically irrelevant to the tasks. On LiveCodeBench, persona prompting were associated with lower correctness scores of the Gemini model by an average of 1.54 percentage points, with one persona exhibiting a decrease of 3.6% (odds ratio = 0.51). In contrast, the accuracy of the GPT-OSS model improved by 3.4 - 5.7% across all personas (odds ratios = 1.8 - 3.0). Maintainability and code style metrics show statistically significant but negligible effect sizes (all Cliff's δ < 0.15), and security vulnerabilities exhibit no systematic persona-specific patterns. Overall, our results show that the presence of demographic information about users is associated with measurable variation in LLM reasoning and code quality even in purely technical tasks, and that these effects hold across models. Our work highlights an under-examined risk in LLM-assisted software development.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

37 of 62 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  7. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  8. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  9. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  10. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 223
  11. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  12. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
25 more
  1. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  2. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  3. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
  4. 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 121
  5. 2023Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Reviewcited by 83
  6. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 62
  7. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  8. 2023Nationality Bias in Text Generationcited by 50
  9. 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
  10. 2023Uncovering and Quantifying Social Biases in Code Generationcited by 25
  11. 2024Bias Unveiled: Investigating Social Bias in LLM-Generated Codecited by 25
  12. 2024Helpful assistant or fruitful facilitator? Investigating how personas affect language model behaviorcited by 21
  13. 2024On the Way to LLM Personalization: Learning to Remember User Conversationscited by 21
  14. 2025Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoningcited by 21
  15. 2025Guidelines for Empirical Studies in Software Engineering involving Large Language Modelscited by 21
  16. 2025Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performancecited by 18
  17. 2024Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTcited by 13
  18. 2023She Elicits Requirements and He Tests: Software Engineering Gender Bias in Large Language Modelscited by 12
  19. 2025Rethinking the Evaluation of Secure Code Generationcited by 10
  20. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 8
  21. 2024Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Coursescited by 6
  22. 2025Is LLM-Generated Code More Maintainable & Reliable than Human-Written Code?cited by 5
  23. 2025What Does a Software Engineer Look Like? Exploring Societal Stereotypes in LLMscited by 4
  24. 2025Characterizing Selective Refusal Bias in Large Language Modelscited by 4
  25. 2024Nigerian Software Engineer or American Data Scientist? GitHub Profile Recruitment Bias in Large Language Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Social Bias in LLM-Generated Code: Benchmark and Mitigation
  2. 2026Measuring and Mitigating Bias in Code Generated by Large Language Models
  3. 2026How Reasoning Shapes Social Bias in LLM-Generated Code?
  4. 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions
  5. 2026Persona Prompting as a Lens on LLM Social Reasoning
  6. 2025BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
  7. 2022Aligning Offline Metrics and Human Judgments of Value for Code Generation Models
  8. 2026More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests