# Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set

arXiv: https://arxiv.org/abs/2501.15634 · HTML: https://mlsanity.com/paper/2501.15634/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 18 of 18)

- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [On the Existence of Simpler Machine Learning Models](https://mlsanity.com/paper/1908.01755/) (2019)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [On Counterfactual Explanations under Predictive Multiplicity](https://mlsanity.com/paper/2006.13132/) (2020)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Empirical observation of negligible fairness-accuracy trade-offs in machine learning for public policy](https://mlsanity.com/paper/2012.02972/) (2020)
- [Predictive Multiplicity in Probabilistic Classification](https://mlsanity.com/paper/2206.01131/) (2022)
- [Consistent Counterfactuals for Deep Models](https://mlsanity.com/paper/2110.03109/) (2021)
- [Leave-one-out Unfairness](https://mlsanity.com/paper/2107.10171/) (2021)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Selective Ensembles for Consistent Predictions](https://mlsanity.com/paper/2111.08230/) (2021)
- [Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models](https://mlsanity.com/paper/1901.03209/) (2019)
- [Affirmative Algorithms: The Legal Grounds for Fairness as Awareness](https://mlsanity.com/paper/2012.14285/) (2020)
- [Reconciling Individual Probability Forecasts](https://mlsanity.com/paper/2209.01687/) (2022)
- [Algorithmic Arbitrariness in Content Moderation](https://mlsanity.com/paper/2402.16979/) (2024)
- [Computing the Collection of Good Models for Rule Lists](https://mlsanity.com/paper/2204.11285/) (2022)

## Built on by (most cited first, 10 of 10)

- [Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems](https://mlsanity.com/paper/2505.18139/) (2025)
- [Fairness and Sparsity within Rashomon sets: Enumeration-Free Exploration and Characterization](https://mlsanity.com/paper/2502.05286/) (2025)
- [ElliCE: Efficient and Provably Robust Algorithmic Recourse via the Rashomon Sets](https://mlsanity.com/paper/2602.07674/) (2026)
- [Selecting for Less Discriminatory Algorithms: A Relational Search Framework for Navigating Fairness-Accuracy Trade-offs in Practice](https://mlsanity.com/paper/2506.01594/) (2025)
- [DIVERSE: Disagreement-Inducing Vector Evolution for Rashomon Set Exploration](https://mlsanity.com/paper/2601.20627/) (2026)
- [What Must a Fairness Audit Report When Demographic Data Is Incomplete?](https://mlsanity.com/paper/2506.23033/) (2025)
- [The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning](https://mlsanity.com/paper/2511.21799/) (2025)
- [Statistical Guarantees in the Search for Less Discriminatory Algorithms](https://mlsanity.com/paper/2512.23943/) (2025)
- [An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness](https://mlsanity.com/paper/2604.23954/) (2026)
- [Model Multiplicity and Predictive Arbitrariness in Recidivism Risk Assessment](https://mlsanity.com/paper/2606.02198/) (2026)

## Abstract (arXiv)

When selecting a model from a set of equally performant models, how much unfairness can you really reduce? Is it important to be intentional about fairness when choosing among this set, or is arbitrarily choosing among the set of ''good'' models good enough? Recent work has highlighted that the phenomenon of model multiplicity-where multiple models with nearly identical predictive accuracy exist for the same task-has both positive and negative implications for fairness, from strengthening the enforcement of civil rights law in AI systems to showcasing arbitrariness in AI decision-making. Despite the enormous implications of model multiplicity, there is little work that explores the properties of sets of equally accurate models, or Rashomon sets, in general. In this paper, we present five main theoretical and methodological contributions which help us to understand the relatively unexplored properties of the Rashomon set, in particular with regards to fairness. Our contributions include methods for efficiently sampling models from this set and techniques for identifying the fairest models according to key fairness metrics such as statistical parity. We also derive the probability that an individual's prediction will be flipped within the Rashomon set, as well as expressions for the set's size and the distribution of error tolerance used across models. These results lead to policy-relevant takeaways, such as the importance of intentionally looking for fair models within the Rashomon set, and understanding which individuals or groups may be more susceptible to arbitrary decisions.