# Is Your Model "MADD"? A Novel Metric to Evaluate Algorithmic Fairness for Predictive Student Models

arXiv: https://arxiv.org/abs/2305.15342 · HTML: https://mlsanity.com/paper/2305.15342/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 10 of 10)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [A survey on datasets for fairness-aware machine learning](https://mlsanity.com/paper/2110.00530/) (2021)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [Algorithmic Fairness in Education](https://mlsanity.com/paper/2007.05443/) (2020)
- [Towards Equity and Algorithmic Fairness in Student Grade Prediction](https://mlsanity.com/paper/2105.06604/) (2021)
- [Evaluation of Fairness Trade-offs in Predicting Student Success](https://mlsanity.com/paper/2007.00088/) (2020)
- [Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?](https://mlsanity.com/paper/2104.12920/) (2021)
- [A Survey on Methods and Metrics for the Assessment of Explainability under the Proposed AI Act](https://mlsanity.com/paper/2110.11168/) (2021)

## Built on by (most cited first, 7 of 7)

- [ABROCA Distributions For Algorithmic Bias Assessment: Considerations Around Interpretation](https://mlsanity.com/paper/2411.19090/) (2024)
- [Exploring Fairness in Educational Data Mining in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2405.16798/) (2024)
- [Counterfactual Fairness Evaluation of Machine Learning Models on Educational Datasets](https://mlsanity.com/paper/2504.11504/) (2025)
- [Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students](https://mlsanity.com/paper/2405.09821/) (2024)
- [Uncertainty-preserving deep knowledge tracing with state-space models](https://mlsanity.com/paper/2407.17427/) (2024)
- [Toward Sufficient Statistical Power in Algorithmic Bias Assessment: A Test for ABROCA](https://mlsanity.com/paper/2501.04683/) (2025)
- [A Fair Post-Processing Method based on the MADD Metric for Predictive Student Models](https://mlsanity.com/paper/2407.05398/) (2024)

## Abstract (arXiv)

Predictive student models are increasingly used in learning environments due to their ability to enhance educational outcomes and support stakeholders in making informed decisions. However, predictive models can be biased and produce unfair outcomes, leading to potential discrimination against some students and possible harmful long-term implications. This has prompted research on fairness metrics meant to capture and quantify such biases. Nonetheless, so far, existing fairness metrics used in education are predictive performance-oriented, focusing on assessing biased outcomes across groups of students, without considering the behaviors of the models nor the severity of the biases in the outcomes. Therefore, we propose a novel metric, the Model Absolute Density Distance (MADD), to analyze models' discriminatory behaviors independently from their predictive performance. We also provide a complementary visualization-based analysis to enable fine-grained human assessment of how the models discriminate between groups of students. We evaluate our approach on the common task of predicting student success in online courses, using several common predictive classification models on an open educational dataset. We also compare our metric to the only predictive performance-oriented fairness metric developed in education, ABROCA. Results on this dataset show that: (1) fair predictive performance does not guarantee fair models' behaviors and thus fair outcomes, (2) there is no direct relationship between data bias and predictive performance bias nor discriminatory behaviors bias, and (3) trained on the same data, models exhibit different discriminatory behaviors, according to different sensitive features too. We thus recommend using the MADD on models that show satisfying predictive performance, to gain a finer-grained understanding on how they behave and to refine models selection and their usage.