# Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling

arXiv: https://arxiv.org/abs/2209.10335 · HTML: https://mlsanity.com/paper/2209.10335/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 63 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [50 Years of Test (Un)fairness: Lessons for Machine Learning](https://mlsanity.com/paper/1811.10104/) (2018)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [Evaluation of Fairness Trade-offs in Predicting Student Success](https://mlsanity.com/paper/2007.00088/) (2020)
- [Argument from Old Man's View: Assessing Social Bias in Argumentation](https://mlsanity.com/paper/2011.12014/) (2020)
- [Supporting Cognitive and Emotional Empathic Writing of Students](https://mlsanity.com/paper/2105.14815/) (2021)
- [A Corpus for Argumentative Writing Support in German](https://mlsanity.com/paper/2010.13674/) (2020)

## Built on by (most cited first, 6 of 6)

- [Fairness and Bias in Multimodal AI: A Survey](https://mlsanity.com/paper/2406.19097/) (2024)
- [Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance](https://mlsanity.com/paper/2311.03311/) (2023)
- [Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study](https://mlsanity.com/paper/2406.14629/) (2024)
- [Identifying Fairness Issues in Automatically Generated Testing Content](https://mlsanity.com/paper/2404.15104/) (2024)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [Student Answer Forecasting: Transformer-Driven Answer Choice Prediction for Language Learning](https://mlsanity.com/paper/2405.20079/) (2024)

## Abstract (arXiv)

Natural Language Processing (NLP) has become increasingly utilized to provide adaptivity in educational applications. However, recent research has highlighted a variety of biases in pre-trained language models. While existing studies investigate bias in different domains, they are limited in addressing fine-grained analysis on educational and multilingual corpora. In this work, we analyze bias across text and through multiple architectures on a corpus of 9,165 German peer-reviews collected from university students over five years. Notably, our corpus includes labels such as helpfulness, quality, and critical aspect ratings from the peer-review recipient as well as demographic attributes. We conduct a Word Embedding Association Test (WEAT) analysis on (1) our collected corpus in connection with the clustered labels, (2) the most common pre-trained German language models (T5, BERT, and GPT-2) and GloVe embeddings, and (3) the language models after fine-tuning on our collected data-set. In contrast to our initial expectations, we found that our collected corpus does not reveal many biases in the co-occurrence analysis or in the GloVe embeddings. However, the pre-trained German language models find substantial conceptual, racial, and gender bias and have significant changes in bias across conceptual and racial axes during fine-tuning on the peer-review data. With our research, we aim to contribute to the fourth UN sustainability goal (quality education) with a novel dataset, an understanding of biases in natural language education data, and the potential harms of not counteracting biases in language models for educational tasks.