# Rashomon effect in Educational Research: Why More is Better Than One for Measuring the Importance of the Variables?

arXiv: https://arxiv.org/abs/2412.12115 · HTML: https://mlsanity.com/paper/2412.12115/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 45 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [A Path to Simpler Models Starts With Noise](https://mlsanity.com/paper/2310.19726/) (2023)
- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [Case Study: Predictive Fairness to Reduce Misdemeanor Recidivism Through Social Service Interventions](https://mlsanity.com/paper/2001.09233/) (2020)
- [An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification](https://mlsanity.com/paper/2405.01557/) (2024)
- [Performance is not enough: the story told by a Rashomon quartet](https://mlsanity.com/paper/2302.13356/) (2023)
- [forester: A Tree-Based AutoML Tool in R](https://mlsanity.com/paper/2409.04789/) (2024)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices](https://mlsanity.com/paper/2506.04553/) (2025)

## Abstract (arXiv)

This study explores how the Rashomon effect influences variable importance in the context of student demographics used for academic outcomes prediction. Our research follows the way machine learning algorithms are employed in Educational Data Mining, focusing on highlighting the so-called Rashomon effect. The study uses the Rashomon set of simple-yet-accurate models trained using decision trees, random forests, light GBM, and XGBoost algorithms with the Open University Learning Analytics Dataset. We found that the Rashomon set improves the predictive accuracy by 2-6%. Variable importance analysis revealed more consistent and reliable results for binary classification than multiclass classification, highlighting the complexity of predicting multiple outcomes. Key demographic variables imd_band and highest_education were identified as vital, but their importance varied across courses, especially in course DDD. These findings underscore the importance of model choice and the need for caution in generalizing results, as different models can lead to different variable importance rankings. The codes for reproducing the experiments are available in the repository: https://anonymous.4open.science/r/JEDM_paper-DE9D.