# LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education

arXiv: https://arxiv.org/abs/2410.14012 · HTML: https://mlsanity.com/paper/2410.14012/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 76 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [Simulating Classroom Education with LLM-Empowered Agents](https://mlsanity.com/paper/2406.19226/) (2024)
- [UnQovering Stereotyping Biases via Underspecified Questions](https://mlsanity.com/paper/2010.02428/) (2020)
- [On Measuring and Mitigating Biased Inferences of Word Embeddings](https://mlsanity.com/paper/1908.09369/) (2019)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2310.05280/) (2023)
- [Comparing Biases and the Impact of Multilingual Training across Multiple Languages](https://mlsanity.com/paper/2305.11242/) (2023)
- [A survey of bias in Machine Learning through the prism of Statistical Parity for the Adult Data Set](https://mlsanity.com/paper/2003.14263/) (2020)
- [Empowering Personalized Learning through a Conversation-based Tutoring System with Student Modeling](https://mlsanity.com/paper/2403.14071/) (2024)
- [Stereotype or Personalization? User Identity Biases Chatbot Recommendations](https://mlsanity.com/paper/2410.05613/) (2024)
- [Uncovering Implicit Gender Bias in Narratives through Commonsense Inference](https://mlsanity.com/paper/2109.06437/) (2021)
- [Beyond Binary Gender Labels: Revealing Gender Biases in LLMs through Gender-Neutral Name Predictions](https://mlsanity.com/paper/2407.05271/) (2024)
- [Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in English, Spanish, and Arabic](https://mlsanity.com/paper/2204.03558/) (2022)
- [From Melting Pots to Misrepresentations: Exploring Harms in Generative AI](https://mlsanity.com/paper/2403.10776/) (2024)
- [User Adaptive Language Learning Chatbots with a Curriculum](https://mlsanity.com/paper/2304.05489/) (2023)

## Built on by (most cited first, 28 of 28)

- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [On the Way to LLM Personalization: Learning to Remember User Conversations](https://mlsanity.com/paper/2411.13405/) (2024)
- [FACET: Teacher-Centred LLM-Based Multi-Agent Systems-Towards Personalized Educational Worksheets](https://mlsanity.com/paper/2508.11401/) (2025)
- [HARBOR: Exploring Persona Dynamics in Multi-Agent Competition](https://mlsanity.com/paper/2502.12149/) (2025)
- [Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs](https://mlsanity.com/paper/2505.15524/) (2025)
- [The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs](https://mlsanity.com/paper/2510.09905/) (2025)
- [Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback](https://mlsanity.com/paper/2511.08225/) (2025)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)
- [AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations](https://mlsanity.com/paper/2607.01934/) (2026)
- [Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback](https://mlsanity.com/paper/2603.12471/) (2026)
- [GenderBench: Evaluation Suite for Gender Biases in LLMs](https://mlsanity.com/paper/2505.12054/) (2025)
- [MIST: Towards Multi-dimensional Implicit BiaS Evaluation of LLMs for Theory of Mind](https://mlsanity.com/paper/2506.14161/) (2025)
- [Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations](https://mlsanity.com/paper/2509.04515/) (2025)
- [A Cross-Lingual Analysis of Bias in Large Language Models Using Romanian History](https://mlsanity.com/paper/2510.02362/) (2025)
- [Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation](https://mlsanity.com/paper/2511.04706/) (2025)
- [Identity-Robust Language Model Generation via Content Integrity Preservation](https://mlsanity.com/paper/2601.09141/) (2026)
- [Implicit Grading Bias in Large Language Models: How Writing Style Affects Automated Assessment Across Math, Programming, and Essay Tasks](https://mlsanity.com/paper/2603.18765/) (2026)
- [Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Courses](https://mlsanity.com/paper/2509.08862/) (2025)
- [HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment](https://mlsanity.com/paper/2510.12217/) (2025)
- [Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens](https://mlsanity.com/paper/2511.03880/) (2025)
- [Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Education](https://mlsanity.com/paper/2601.14506/) (2026)
- [Probing Cultural Signals in Large Language Models through Author Profiling](https://mlsanity.com/paper/2603.16749/) (2026)
- [Sociodemographic Biases in Educational Counselling by Large Language Models](https://mlsanity.com/paper/2604.25932/) (2026)
- [Just Ask for a Table: A Thirty-Token User Prompt Defeats Sponsored Recommendations in Twelve LLMs](https://mlsanity.com/paper/2605.12772/) (2026)
- [Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents](https://mlsanity.com/paper/2606.01584/) (2026)
- [The Paternalistic Filter: Epistemic Injustice and Differential Refusal in LLM-Mediated History Education for Marginalized Romanian Students](https://mlsanity.com/paper/2607.11292/) (2026)
- [Auditing Alignment Controllability in LLMs via Political Axes](https://mlsanity.com/paper/2607.23519/) (2026)
- [EduFair-Bench: Evaluating Pedagogical Fairness of LLM Tutors Across Student Demographics](https://mlsanity.com/paper/2609.12949/) (2026)

## Abstract (arXiv)

With the increasing adoption of large language models (LLMs) in education, concerns about inherent biases in these models have gained prominence. We evaluate LLMs for bias in the personalized educational setting, specifically focusing on the models' roles as "teachers." We reveal significant biases in how models generate and select educational content tailored to different demographic groups, including race, ethnicity, sex, gender, disability status, income, and national origin. We introduce and apply two bias score metrics--Mean Absolute Bias (MAB) and Maximum Difference Bias (MDB)--to analyze 9 open and closed state-of-the-art LLMs. Our experiments, which utilize over 17,000 educational explanations across multiple difficulty levels and topics, uncover that models potentially harm student learning by both perpetuating harmful stereotypes and reversing them. We find that bias is similar for all frontier models, with the highest MAB along income levels while MDB is highest relative to both income and disability status. For both metrics, we find the lowest bias exists for sex/gender and race/ethnicity.