mlsanityData to 14 Oct 2026

Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Education

arXiv 2601.14506 ↗First posted 20 Jan 2026cs.CY cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are increasingly deployed in STEM education for personalized instruction and feedback across institutions in high- and low-income countries. These systems are designed to adapt content to student needs, but whether they adapt based on demonstrated ability or demographic signals remains untested at scale. Here we establish that LLM-generated STEM content systematically disadvantages marginalized student profiles across two cultural contexts, with the gap between the most privileged and most marginalized profiles reaching 2.55 grade levels. We audited four LLMs (Qwen 2.5-32B-Instruct, GPT-4o, GPT-4o-mini, GPT-OSS 20B) using synthetic profiles crossing dimensions specific to Indian education (caste, medium of instruction, college tier) and American education (race, HBCU attendance, school type), alongside income, gender, and disability, across ranking and generation tasks with FDR-corrected significance testing and SHAP feature attribution. Income produces significant effects across every model and context, medium of instruction drives the largest single effect in the Indian context, and disability status triggers simpler explanations. Effects compound non-additively: marginalization across multiple dimensions produces gaps larger than any single dimension predicts, and biases persist within elite institutions. Bias is consistent across all four architectures and persists through model selection, making intersectional, cross-cultural auditing a structural requirement before deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 77 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

34 of 77 references.

References · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  5. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  9. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  10. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  11. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 104
  12. 2020Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligencecited by 101
22 more
  1. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 97
  2. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Modelscited by 77
  3. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 60
  4. 2023AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asiacited by 54
  5. 2024Stereotype or Personalization? User Identity Biases Chatbot Recommendationscited by 34
  6. 2023'Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusioncited by 33
  7. 2024Identifying and Improving Disability Bias in GPT-Based Resume Screeningcited by 28
  8. 2024LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Educationcited by 28
  9. 2024Exploring Safety-Utility Trade-Offs in Personalized Language Modelscited by 27
  10. 2024"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversationscited by 25
  11. 2023An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Modelscited by 24
  12. 2023Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Modelscited by 18
  13. 2025How Deep Is Representational Bias in LLMs? The Cases of Caste and Religioncited by 14
  14. 2021Towards Equity and Algorithmic Fairness in Student Grade Predictioncited by 13
  15. 2025DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysiscited by 12
  16. 2025Detecting Prefix Bias in LLM-based Reward Modelscited by 11
  17. 2023"It's not like Jarvis, but it's pretty close!" -- Examining ChatGPT's Usage among Undergraduate Students in Computer Sciencecited by 10
  18. 2024As an AI Language Model, "Yes I Would Recommend Calling the Police": Norm Inconsistency in LLM Decision-Makingcited by 10
  19. 2024Interpretations, Representations, and Stereotypes of Caste within Text-to-Image Generatorscited by 10
  20. 2022When Personalization Harms: Reconsidering the Use of Group Attributes in Predictioncited by 6
  21. 2025Adultification Bias in LLMs and Text-to-Image Modelscited by 5
  22. 2025Documenting Patterns of Exoticism of Marginalized Populations within Text-to-Image Generatorscited by 3

Built on by

Not cited yet.