Why are there many equally good models? An Anatomy of the Rashomon Effect
Not on the map. Why?
Abstract (arXiv)
The Rashomon effect -- the existence of multiple, distinct models that achieve nearly equivalent predictive performance -- has emerged as a fundamental phenomenon in modern machine learning and statistics. In this paper, we explore the causes underlying the Rashomon effect, organizing them into three categories: statistical sources arising from finite samples and noise in the data-generating process; structural sources arising from non-convexity of optimization objectives and unobserved variables that create fundamental non-identifiability; and procedural sources arising from limitations of optimization algorithms and deliberate restrictions to suboptimal model classes. We synthesize insights from machine learning, statistics, and optimization literature to provide a unified framework for understanding why the multiplicity of good models arises. A key distinction emerges: statistical multiplicity diminishes with more data, structural multiplicity persists asymptotically and cannot be resolved without different data or additional assumptions, and procedural multiplicity reflects choices made by practitioners. Beyond characterizing causes, we discuss both the challenges and opportunities presented by the Rashomon effect, including implications for inference, interpretability, fairness, and decision-making under uncertainty.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 74 references that are in the map. First posted on arXiv 11 Jan 2026.
Builds on
37 of 74 references.
References · 37
- 2017Attention Is All You Needcited by 60,361
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
- 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
- 2017Algorithmic decision making and the cost of fairnesscited by 403
- 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 384
- 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 373
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
- 2018Essentially No Barriers in Neural Network Energy Landscapecited by 330
25 more
- 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 290
- 2014The Loss Surfaces of Multilayer Networkscited by 280
- 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
- 2019Predictive Multiplicity in Classificationcited by 166
- 2015Path-SGD: Path-Normalized Optimization in Deep Neural Networkscited by 120
- 2015Best Subset Selection via a Modern Optimization Lenscited by 100
- 2019On the Existence of Simpler Machine Learning Modelscited by 89
- 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 81
- 2021Characterizing Fairness Over the Set of Good Models Under Selective Labelscited by 65
- 2024Amazing Things Come From Having Many Good Modelscited by 55
- 2022Rashomon Capacity: A Metric for Predictive Multiplicity in Classificationcited by 47
- 2015Monotonic Calibrated Interpolated Look-Up Tablescited by 35
- 2023A Path to Simpler Models Starts With Noisecited by 33
- 2023Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classificationcited by 32
- 2016Assessing and tuning brain decoders: cross-validation, caveats, and guidelinescited by 30
- 2023Exploring and Interacting with the Set of Good Sparse Generalized Additive Modelscited by 30
- 2023The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importancecited by 29
- 2022FasterRisk: Fast and Accurate Interpretable Risk Scorescited by 21
- 2021GAM Changer: Editing Generalized Additive Models with Interactive Visualizationcited by 18
- 2020Predictive and Causal Implications of using Shapley Value for Model Interpretationcited by 16
- 2018A Theory of Statistical Inference for Ensuring the Robustness of Scientific Resultscited by 13
- 2022TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualizationcited by 13
- 2024Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Populationcited by 6
- 2025Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effectcited by 3
- 2025Parameter-Efficient Construction of the Rashomon Slice for Concept Bottleneck Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.