# EvoXplain: When Machine Learning Models Agree on Predictions but Disagree on Why -- Measuring Mechanistic Multiplicity Across Training Runs

arXiv: https://arxiv.org/abs/2512.22240 · HTML: https://mlsanity.com/paper/2512.22240/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 57 references that are in the map.
- First posted on arXiv 23 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS)](https://mlsanity.com/paper/2102.01564/) (2021)
- [Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models](https://mlsanity.com/paper/2001.11757/) (2020)
- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [Mechanistic Interpretability as Statistical Estimation: A Variance Analysis](https://mlsanity.com/paper/2510.00845/) (2025)
- [The BIG Argument for AI Safety Cases](https://mlsanity.com/paper/2503.11705/) (2025)
- [Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoML](https://mlsanity.com/paper/2507.14744/) (2025)
- [Breast cancer detection using artificial intelligence techniques: A systematic literature review](https://mlsanity.com/paper/2203.04308/) (2022)
- [Modular Jets for Supervised Pipelines: Diagnosing Mirage vs Identifiability](https://mlsanity.com/paper/2512.05638/) (2025)

## Built on by (most cited first, 1 of 1)

- [Hypothesis Class Determines Explanation: Why Accurate Models Disagree on Feature Attribution](https://mlsanity.com/paper/2603.15821/) (2026)

## Abstract (arXiv)

Machine learning models are primarily judged by predictive performance, especially in applied genomics, where explanations are read as biological findings. In practice, reported gene panels are stabilised by averaging, ranking, or taking consensus over the many models a pipeline produces across cross-validation folds, tuning grids, and repeated runs. This raises an overlooked question: when two models achieve high accuracy, do they rely on the same internal logic, or reach the same outcome via different mechanisms? We introduce EvoXplain, a diagnostic framework that measures whether a pipeline's explanation is uniquely determined across repeated training and model selection. Rather than analysing a single trained model, EvoXplain treats explanations as samples drawn from the training and model selection pipeline itself, without aggregating predictions or constructing ensembles, and examines whether they form a single coherent explanatory basin or separate into multiple structured basins. We evaluate EvoXplain on a TCGA pan-cancer cohort and a within-cancer breast-cancer subtype task, using elastic-net Logistic Regression and gradient-boosted trees. Although all models reach about 98% accuracy, explanation structure differs across pipelines. Holding the data split fixed and varying only the regularisation strength, equally accurate Logistic Regression models separate into a few discrete, reproducible basins that recur across 100 data splits and carry distinct biological content, while the gradient-boosted pipeline converges to one basin. The same multiplicity appears within a single cancer subtype, from the ordinary tuning step alone. EvoXplain makes explanatory structure visible, revealing when an averaged consensus corresponds to no single trained model, and reframes interpretability as a property of the training pipeline rather than of any single model.