# On the Existence of Simpler Machine Learning Models

arXiv: https://arxiv.org/abs/1908.01755 · HTML: https://mlsanity.com/paper/1908.01755/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2019.

## Built on by (most cited first, 30 of 89)

- [Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?](https://mlsanity.com/paper/2211.13972/) (2022)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [A Path to Simpler Models Starts With Noise](https://mlsanity.com/paper/2310.19726/) (2023)
- [Exploring and Interacting with the Set of Good Sparse Generalized Additive Models](https://mlsanity.com/paper/2303.16047/) (2023)
- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning](https://mlsanity.com/paper/2306.15786/) (2023)
- [Position: Explain to Question not to Justify](https://mlsanity.com/paper/2402.13914/) (2024)
- [Arbitrary Decisions are a Hidden Cost of Differentially Private Training](https://mlsanity.com/paper/2302.14517/) (2023)
- [The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions](https://mlsanity.com/paper/2304.10655/) (2023)
- [Learning Optimal Fair Classification Trees: Trade-offs Between Interpretability, Fairness, and Accuracy](https://mlsanity.com/paper/2201.09932/) (2022)
- [Recourse under Model Multiplicity via Argumentative Ensembling (Technical Report)](https://mlsanity.com/paper/2312.15097/) (2023)
- [Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time](https://mlsanity.com/paper/2503.01087/) (2025)
- [Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set](https://mlsanity.com/paper/2110.13369/) (2021)
- [Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data](https://mlsanity.com/paper/2308.11446/) (2023)
- [Interpretable and Fair Boolean Rule Sets via Column Generation](https://mlsanity.com/paper/2111.08466/) (2021)
- [Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2307.14239/) (2023)
- [Efficient Exploration of the Rashomon Set of Rule Set Models](https://mlsanity.com/paper/2406.03059/) (2024)
- [Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set](https://mlsanity.com/paper/2501.15634/) (2025)
- [What Constitutes a Less Discriminatory Algorithm?](https://mlsanity.com/paper/2412.18138/) (2024)
- [Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems](https://mlsanity.com/paper/2505.18139/) (2025)
- [Credal Prediction based on Relative Likelihood](https://mlsanity.com/paper/2505.22332/) (2025)
- [Investigating the Impact of Balancing, Filtering, and Complexity on Predictive Multiplicity: A Data-Centric Perspective](https://mlsanity.com/paper/2412.09712/) (2024)
- [Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Population](https://mlsanity.com/paper/2401.14512/) (2024)
- [On f-Divergence Principled Domain Adaptation: An Improved Framework](https://mlsanity.com/paper/2402.01887/) (2024)
- [A Market for Accuracy: Classification under Competition](https://mlsanity.com/paper/2502.18052/) (2025)
- [Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoML](https://mlsanity.com/paper/2507.14744/) (2025)
- [SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumeration](https://mlsanity.com/paper/2511.03344/) (2025)
- [Exploring the cloud of feature interaction scores in a Rashomon set](https://mlsanity.com/paper/2305.10181/) (2023)

## Abstract (arXiv)

It is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is generally NP-hard. We often do not know whether the search for a simpler model will be worthwhile, and thus we do not go to the trouble of searching for one. In this work, we ask an important practical question: can accurate-yet-simple models be proven to exist, or shown likely to exist, before explicitly searching for them? We hypothesize that there is an important reason that simple-yet-accurate models often do exist. This hypothesis is that the size of the Rashomon set is often large, where the Rashomon set is the set of almost-equally-accurate models from a function class. If the Rashomon set is large, it contains numerous accurate models, and perhaps at least one of them is the simple model we desire. In this work, we formally present the Rashomon ratio as a new gauge of simplicity for a learning problem, depending on a function class and a data set. The Rashomon ratio is the ratio of the volume of the set of accurate models to the volume of the hypothesis space, and it is different from standard complexity measures from statistical learning theory. Insight from studying the Rashomon ratio provides an easy way to check whether a simpler model might exist for a problem before finding it, namely whether several different machine learning methods achieve similar performance on the data. In that sense, the Rashomon ratio is a powerful tool for understanding why and when an accurate-yet-simple model might exist. If, as we hypothesize in this work, many real-world data sets admit large Rashomon sets, the implications are vast: it means that simple or interpretable models may often be used for high-stakes decisions without losing accuracy.