# Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback

arXiv: https://arxiv.org/abs/2511.08225 · HTML: https://mlsanity.com/paper/2511.08225/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 56 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 21 of 21)

- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases](https://mlsanity.com/paper/2006.03955/) (2020)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation](https://mlsanity.com/paper/2402.12649/) (2024)
- [Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs](https://mlsanity.com/paper/2401.06431/) (2024)
- [LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education](https://mlsanity.com/paper/2410.14012/) (2024)
- [Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge](https://mlsanity.com/paper/2504.07887/) (2025)
- [When Text Embedding Meets Large Language Model: A Comprehensive Survey](https://mlsanity.com/paper/2412.09165/) (2024)
- [Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study](https://mlsanity.com/paper/2504.05570/) (2025)
- [Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation](https://mlsanity.com/paper/2407.08441/) (2024)
- [Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation](https://mlsanity.com/paper/2505.08546/) (2025)
- [EuroGEST: Investigating gender stereotypes in multilingual language models](https://mlsanity.com/paper/2506.03867/) (2025)

## Built on by (most cited first, 5 of 5)

- [Disentangling Learning from Judgment: Representation Learning for Open Response Analytics](https://mlsanity.com/paper/2512.23941/) (2025)
- [Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education](https://mlsanity.com/paper/2512.23036/) (2025)
- [Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling](https://mlsanity.com/paper/2604.08263/) (2026)
- [Understanding Teacher Revisions of Large Language Model-Generated Feedback](https://mlsanity.com/paper/2603.27806/) (2026)
- [Evaluating Feedback Focus and Pedagogical Adaptivity in LLM-Generated Feedback on Student Writing](https://mlsanity.com/paper/2609.28026/) (2026)

## Abstract (arXiv)

As teachers increasingly turn to GenAI in their educational practice, we need robust methods to benchmark large language models (LLMs) for pedagogical purposes. This article presents an embedding-based benchmarking framework to detect bias in LLMs in the context of formative feedback. Using 600 authentic student essays from the AES 2.0 corpus, we constructed controlled counterfactuals along two dimensions: (i) implicit cues via lexicon-based swaps of gendered terms within essays, and (ii) explicit cues via gendered author background in the prompt. We investigated six representative LLMs (i.e. GPT-5 mini, GPT-4o mini, DeepSeek-R1, DeepSeek-R1-Qwen, Gemini 2.5 Pro, Llama-3-8B). We first quantified the response divergence with cosine and Euclidean distances over sentence embeddings, then assessed significance via permutation tests, and finally, visualised structure using dimensionality reduction. In all models, implicit manipulations reliably induced larger semantic shifts for male-female counterfactuals than for female-male. Only the GPT and Llama models showed sensitivity to explicit gender cues. These findings show that even state-of-the-art LLMs exhibit asymmetric semantic responses to gender substitutions, suggesting persistent gender biases in feedback they provide learners. Qualitative analyses further revealed consistent linguistic differences (e.g., more autonomy-supportive feedback under male cues vs. more controlling feedback under female cues). We discuss implications for fairness auditing of pedagogical GenAI, propose reporting standards for counterfactual evaluation in learning analytics, and outline practical guidance for prompt design and deployment to safeguard equitable feedback.