# When Interpretability Is Unequally Distributed: Fairness in Hybrid Interpretable Models

arXiv: https://arxiv.org/abs/2605.28626 · HTML: https://mlsanity.com/paper/2605.28626/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 24 of 24)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Counterfactual Explanations Can Be Manipulated](https://mlsanity.com/paper/2106.02666/) (2021)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Supersparse Linear Integer Models for Optimized Medical Scoring Systems](https://mlsanity.com/paper/1502.04269/) (2015)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Equalizing Recourse across Groups](https://mlsanity.com/paper/1909.03166/) (2019)
- [The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations](https://mlsanity.com/paper/2205.03295/) (2022)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Universal Adversarial Perturbations: A Survey](https://mlsanity.com/paper/2005.08087/) (2020)
- [Fairness and Explainability: Bridging the Gap Towards Fair Model Explanations](https://mlsanity.com/paper/2212.03840/) (2022)
- [A Theory of Statistical Inference for Ensuring the Robustness of Scientific Results](https://mlsanity.com/paper/1804.08646/) (2018)
- [Computing the Collection of Good Models for Rule Lists](https://mlsanity.com/paper/2204.11285/) (2022)
- [Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity](https://mlsanity.com/paper/2401.16088/) (2024)
- [SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumeration](https://mlsanity.com/paper/2511.03344/) (2025)
- [Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods](https://mlsanity.com/paper/2303.04437/) (2023)
- [Fairness and Sparsity within Rashomon sets: Enumeration-Free Exploration and Characterization](https://mlsanity.com/paper/2502.05286/) (2025)
- [Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods](https://mlsanity.com/paper/2505.01198/) (2025)
- [Fair Recourse for All: Ensuring Individual and Group Fairness in Counterfactual Explanations](https://mlsanity.com/paper/2601.20449/) (2026)

## Abstract (arXiv)

Hybrid interpretable models combine a transparent component with a black-box model by assigning some examples to the former and deferring the rest to the latter. While this design enables flexible tradeoffs between accuracy and interpretability, it also raises a distinct procedural fairness concern: some demographic groups may systematically receive interpretable decisions, while others are disproportionately routed to a black box. We formalize this issue as Interpretability Coverage Disparity (ICD), a demographic-parity-style measure applied to the routing decision of hybrid interpretable models. Using tools from predictive multiplicity, we study ICD across four hybrid interpretable learning methods, three standard fairness benchmark datasets, and multiple sensitive attributes. Our experiments reveal substantial ICD in intermediate transparency regimes, where both the interpretable and black-box components are actively used. We further show that simple coverage-disparity constraints can significantly reduce ICD in exact hybrid learning methods, with marginal impact on accuracy and sparsity. In several settings, ICD mitigation also improves standard algorithmic fairness metrics. These results show that hybrid interpretable models should be audited not only for predictive fairness, but also for how they allocate interpretability across individuals and groups.