# Intervention Efficiency and Perturbation Validation Framework: Capacity-Aware and Robust Clinical Model Selection under the Rashomon Effect

arXiv: https://arxiv.org/abs/2511.14317 · HTML: https://mlsanity.com/paper/2511.14317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 66 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Builds on (most cited first, 10 of 10)

- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Interpretability of machine learning based prediction models in healthcare](https://mlsanity.com/paper/2002.08596/) (2020)
- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Cross-validation failure: small sample sizes lead to large error bars](https://mlsanity.com/paper/1706.07581/) (2017)
- [Generalizability vs. Robustness: Adversarial Examples for Medical Imaging](https://mlsanity.com/paper/1804.00504/) (2018)
- [Nested cross-validation when selecting classifiers is overzealous for most practical applications](https://mlsanity.com/paper/1809.09446/) (2018)
- [Model Validation Using Mutated Training Labels: An Exploratory Study](https://mlsanity.com/paper/1905.10201/) (2019)
- [Improving the compromise between accuracy, interpretability and personalization of rule-based machine learning in medical problems](https://mlsanity.com/paper/2106.07827/) (2021)
- [Interpretable Predictive Models for Healthcare via Rational Logistic Regression](https://mlsanity.com/paper/2411.03224/) (2024)

## Abstract (arXiv)

In clinical machine learning, the coexistence of multiple models with comparable performance (a manifestation of the Rashomon Effect) poses fundamental challenges for trustworthy deployment and evaluation. Small, imbalanced, and noisy datasets, coupled with high-dimensional and weakly identified clinical features, amplify this multiplicity and make conventional validation schemes unreliable. As a result, selecting among equally performing models becomes uncertain, particularly when resource constraints and operational priorities are not considered by conventional metrics like F1 score. To address these issues, we propose two complementary tools for robust model assessment and selection: Intervention Efficiency (IE) and the Perturbation Validation Framework (PVF). IE is a capacity-aware metric that quantifies how efficiently a model identifies actionable true positives when only limited interventions are feasible, thereby linking predictive performance with clinical utility. PVF introduces a structured approach to assess the stability of models under data perturbations, identifying models whose performance remains most invariant across noisy or shifted validation sets. Empirical results on synthetic and real-world healthcare datasets show that using these tools facilitates the selection of models that generalize more robustly and align with capacity constraints, offering a new direction for tackling the Rashomon Effect in clinical settings.