mlsanityData to 7 Oct 2026

Amazing Things Come From Having Many Good Models

arXiv 2407.04846 ↗First posted 5 Jul 2024cs.LG cs.AIcited by 52

Not on the map. Why?

Abstract (arXiv)

The Rashomon Effect, coined by Leo Breiman, describes the phenomenon that there exist many equally good predictive models for the same dataset. This phenomenon happens for many real datasets and when it does, it sparks both magic and consternation, but mostly magic. In light of the Rashomon Effect, this perspective piece proposes reshaping the way we think about machine learning, particularly for tabular data problems in the nondeterministic (noisy) setting. We address how the Rashomon Effect impacts (1) the existence of simple-yet-accurate models, (2) flexibility to address user preferences, such as fairness and monotonicity, without losing performance, (3) uncertainty in predictions, fairness, and explanations, (4) reliable variable importance, (5) algorithm choice, specifically, providing advanced knowledge of which algorithms might be suitable for a given problem, and (6) public policy. We also discuss a theory of when the Rashomon Effect occurs and why. Our goal is to illustrate how the Rashomon Effect can have a massive impact on the use of machine learning for complex problems in society.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

32 of 59 references.

References · 32

  1. 2018Sanity Checks for Saliency Mapscited by 1,149
  2. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  3. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  4. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  5. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  6. 2019Predictive Multiplicity in Classificationcited by 162
  7. 2020Generalized and Scalable Optimal Sparse Decision Treescited by 98
  8. 2019On the Existence of Simpler Machine Learning Modelscited by 87
  9. 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
  10. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  11. 2021Characterizing Fairness Over the Set of Good Models Under Selective Labelscited by 65
  12. 2018The age of secrecy and unfairness in recidivism predictioncited by 53
20 more
  1. 2022Rashomon Capacity: A Metric for Predictive Multiplicity in Classificationcited by 46
  2. 2015Interpretable Classification Models for Recidivism Predictioncited by 44
  3. 2020Empirical observation of negligible fairness-accuracy trade-offs in machine learning for public policycited by 44
  4. 2022Predictive Multiplicity in Probabilistic Classificationcited by 44
  5. 2020In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Predictioncited by 42
  6. 2021Fast Sparse Decision Tree Optimization via Reference Ensemblescited by 37
  7. 2023A Path to Simpler Models Starts With Noisecited by 33
  8. 2023Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classificationcited by 31
  9. 2023Exploring and Interacting with the Set of Good Sparse Generalized Additive Modelscited by 30
  10. 2023The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importancecited by 29
  11. 2022FasterRisk: Fast and Accurate Interpretable Risk Scorescited by 21
  12. 2022Fast Sparse Classification for Generalized Linear and Additive Modelscited by 19
  13. 2021GAM Changer: Editing Generalized Additive Models with Interactive Visualizationcited by 18
  14. 2021A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanationscited by 16
  15. 2018A Theory of Statistical Inference for Ensuring the Robustness of Scientific Resultscited by 13
  16. 2022TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualizationcited by 13
  17. 2022Computing the Collection of Good Models for Rule Listscited by 12
  18. 2023OKRidge: Scalable Optimal k-Sparse Ridge Regressioncited by 7
  19. 2024Sparse and Faithful Explanations Without Sparse Modelscited by 5
  20. 2023Fast and Interpretable Mortality Risk Scores for Critical Care Patientscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 30; 2026: 18.

52 papers.

Sort:

By area:

  1. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  2. 2025Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Timecited by 13
  3. 2025Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Setcited by 10
  4. 2024What Constitutes a Less Discriminatory Algorithm?cited by 8
  5. 2025Near Optimal Decision Trees in a SPLIT Secondcited by 8
  6. 2025Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systemscited by 8
  7. 2025SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumerationcited by 5
  8. 2025The Role of Hyperparameters in Predictive Multiplicitycited by 4
  9. 2025Leveraging Predictive Equivalence in Decision Treescited by 4
  10. 2025PCS Workflow for Veridical Data Science in the Age of AIcited by 4
  11. 2025Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effectcited by 3
  12. 2025All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogatescited by 3
38 more
  1. 2024Influence-based Attributions can be Manipulatedcited by 2
  2. 2025Rashomon perspective for measuring uncertainty in the survival predictive maintenance modelscited by 2
  3. 2025Uncertainty Quantification as a Principled Foundation for Explainable Artificial Intelligence: A Case Study of Counterfactual Explanationscited by 2
  4. 2025Unique Rashomon Sets for Robust Active Learningcited by 2
  5. 2025Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practicescited by 2
  6. 2026ElliCE: Efficient and Provably Robust Algorithmic Recourse via the Rashomon Setscited by 2
  7. 2026Using predictive multiplicity to measure individual performance within the AI Actcited by 2
  8. 2024DPA: A one-stop metric to measure bias amplification in classification datasetscited by 1
  9. 2025Predictive Multiplicity in Survival Models: A Method for Quantifying Model Uncertainty in Predictive Maintenance Applicationscited by 1
  10. 2025Reconsidering Fairness Through Unawareness From the Perspective of Model Multiplicitycited by 1
  11. 2025Leo Breiman, the Rashomon Effect, and the Occam Dilemmacited by 1
  12. 2025Towards the Formalization of a Trustworthy AI for Mining Interpretable Models explOiting Sophisticated Algorithmscited by 1
  13. 2025Complete Evidence Extraction with Model Ensembles: A Case Study on Medical Codingcited by 1
  14. 2025Parameter-Efficient Construction of the Rashomon Slice for Concept Bottleneck Modelscited by 1
  15. 2026Why are there many equally good models? An Anatomy of the Rashomon Effectcited by 1
  16. 2026Resolving Predictive Multiplicity for the Rashomon Setcited by 1
  17. 2026CASHomon Sets: Efficient Rashomon Sets Across Multiple Model Classes and their Hyperparameterscited by 1
  18. 2026Robust Ambiguity Detection (RAD) From Model- and Feature-Space Consistencycited by 1
  19. 2024EXAGREE: Mitigating Explanation Disagreement with Stakeholder-Aligned Modelscited by 0
  20. 2025Embracing Ambiguity: Bayesian Nonparametrics and Stakeholder Participation for Ambiguity-Aware Safety Evaluationcited by 0
  21. 2025Navigating the Rashomon Effect: How Personalization Can Help Adjust Interpretable Machine Learning Models to Individual Userscited by 0
  22. 2025On Arbitrary Predictions from Equally Valid Modelscited by 0
  23. 2025VAR: Visual Analysis for Rashomon Set of Machine Learning Models' Performancecited by 0
  24. 2025Data as a Lever: A Neighbouring Datasets Perspective on Predictive Multiplicitycited by 0
  25. 2025Intervention Efficiency and Perturbation Validation Framework: Capacity-Aware and Robust Clinical Model Selection under the Rashomon Effectcited by 0
  26. 2025The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learningcited by 0
  27. 2025Predicting and Interpolating Spatiotemporal Environmental Data: A Case Study of Groundwater Storage in Bangladeshcited by 0
  28. 2025Statistical Guarantees in the Search for Less Discriminatory Algorithmscited by 0
  29. 2026Analyzing the Temporal Factors for Anxiety and Depression Symptoms with the Rashomon Perspectivecited by 0
  30. 2026Rashomon Sets and Model Multiplicity in Federated Learningcited by 0
  31. 2026Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Seriescited by 0
  32. 2026Horizon-Constrained Rashomon Sets for Chaotic Forecastingcited by 0
  33. 2026Explainable AI Isn't Enough! Rethinking Algorithmic Contestabilitycited by 0
  34. 2026From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal k-Sparse GLMscited by 0
  35. 2026Model Multiplicity and Predictive Arbitrariness in Recidivism Risk Assessmentcited by 0
  36. 2026Is Model Instability just Noise to be Tolerated or a Property that can be Managed?cited by 0
  37. 2026Do Judges Behave Like Algorithms?cited by 0
  38. 2026TetrisCNN for interpretable detection of phases of matter from experimental quantum simulator datacited by 0

50 most cited of 52 All 52 (JSON)