# Bringing a Ruler Into the Black Box: Uncovering Feature Impact from Individual Conditional Expectation Plots

arXiv: https://arxiv.org/abs/2109.02724 · HTML: https://mlsanity.com/paper/2109.02724/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 6 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation](https://mlsanity.com/paper/2006.14284/) (2020)
- [Engineering Safety in Machine Learning](https://mlsanity.com/paper/1601.04126/) (2016)
- [Nonparametric Feature Impact and Importance](https://mlsanity.com/paper/2006.04750/) (2020)

## Built on by (most cited first, 2 of 2)

- [Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities](https://mlsanity.com/paper/2308.08407/) (2023)
- [TRIVEA: Transparent Ranking Interpretation using Visual Explanation of Black-Box Algorithmic Rankers](https://mlsanity.com/paper/2308.14622/) (2023)

## Abstract (arXiv)

As machine learning systems become more ubiquitous, methods for understanding and interpreting these models become increasingly important. In particular, practitioners are often interested both in what features the model relies on and how the model relies on them--the feature's impact on model predictions. Prior work on feature impact including partial dependence plots (PDPs) and Individual Conditional Expectation (ICE) plots has focused on a visual interpretation of feature impact. We propose a natural extension to ICE plots with ICE feature impact, a model-agnostic, performance-agnostic feature impact metric drawn out from ICE plots that can be interpreted as a close analogy to linear regression coefficients. Additionally, we introduce an in-distribution variant of ICE feature impact to vary the influence of out-of-distribution points as well as heterogeneity and non-linearity measures to characterize feature impact. Lastly, we demonstrate ICE feature impact's utility in several tasks using real-world data.