# The Impact of Stochasticity on the Rashomon Effect in Machine Learning

arXiv: https://arxiv.org/abs/2609.32934 · HTML: https://mlsanity.com/paper/2609.32934/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [Aligning Faithful Interpretations with their Social Attribution](https://mlsanity.com/paper/2006.01067/) (2020)
- [Order in the Court: Explainable AI Methods Prone to Disagreement](https://mlsanity.com/paper/2105.03287/) (2021)
- [A Path to Simpler Models Starts With Noise](https://mlsanity.com/paper/2310.19726/) (2023)
- [An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning](https://mlsanity.com/paper/2306.15786/) (2023)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Agree to Disagree: When Deep Learning Models With Identical Architectures Produce Distinct Explanations](https://mlsanity.com/paper/2105.06791/) (2021)
- [Explanation sensitivity to the randomness of large language models: the case of journalistic text classification](https://mlsanity.com/paper/2410.05085/) (2024)
- [A general approach to compute the relevance of middle-level input features](https://mlsanity.com/paper/2010.08639/) (2020)
- [Can we Agree? On the Rash=omon Effect and the Reliability of Post-Hoc Explainable AI](https://mlsanity.com/paper/2308.07247/) (2023)
- [On the Rashomon ratio of infinite hypothesis sets](https://mlsanity.com/paper/2404.17746/) (2024)

## Abstract (arXiv)

Neural network training is inherently stochastic, with factors such as weight initialization leading to distinct models despite comparable predictive performance. This phenomenon is commonly associated with the Rashomon effect, which describes the existence of multiple near-optimal models for the same task. Although the Rashomon effect has received increasing attention, it remains unclear whether different sources of training stochasticity contribute similarly or differently to its manifestations. In this work, we present an empirical study of the Rashomon phenomenon along three complementary dimensions: solution-space multiplicity, predictive multiplicity, and decision-basis multiplicity. These dimensions are quantified through the size of the empirical Rashomon set, predictive ambiguity, and agreement between XAI attribution maps, respectively. By independently controlling three standard sources of stochasticity, namely weight initialization, mini-batch data ordering, and dropout, we isolate their respective contributions to each dimension of the Rashomon phenomenon. Experiments on tabular and image classification benchmarks reveal that these sources affect the three dimensions in different ways. In particular, larger empirical Rashomon sets do not necessarily correspond to greater predictive disagreement or lower explanation agreement, indicating that solution-space, predictive, and decision-basis multiplicity capture complementary rather than interchangeable aspects of the Rashomon effect. Overall, our results show that training stochasticity influences not only predictive performance but also the stability of predictions and explanations, highlighting the importance of identifying the specific sources of stochasticity responsible for different manifestations of the Rashomon phenomenon when assessing the reliability, reproducibility, and interpretability of neural network models.