# Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing

arXiv: https://arxiv.org/abs/2606.15914 · HTML: https://mlsanity.com/paper/2606.15914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models](https://mlsanity.com/paper/2408.03907/) (2024)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [How LLMs Distort Our Written Language](https://mlsanity.com/paper/2603.18161/) (2026)
- [Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance](https://mlsanity.com/paper/2311.03311/) (2023)
- [Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts](https://mlsanity.com/paper/2410.11084/) (2024)
- [The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?](https://mlsanity.com/paper/2410.04699/) (2024)
- [The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs](https://mlsanity.com/paper/2407.18786/) (2024)
- [Gender Bias in LLM-generated Interview Responses](https://mlsanity.com/paper/2410.20739/) (2024)
- [No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy](https://mlsanity.com/paper/2509.04404/) (2025)
- [Quantifying Cognitive Bias Induction in LLM-Generated Content](https://mlsanity.com/paper/2507.03194/) (2025)
- [When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing](https://mlsanity.com/paper/2409.20390/) (2024)
- [Exploring and Mitigating Gender Bias in Encoder-Based Transformer Models](https://mlsanity.com/paper/2511.00519/) (2025)

## Abstract (arXiv)

Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants (N = 123) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.