# Transforming Feature Space to Interpret Machine Learning Models

arXiv: https://arxiv.org/abs/2104.04295 · HTML: https://mlsanity.com/paper/2104.04295/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance](https://mlsanity.com/paper/1905.03151/) (2019)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach](https://mlsanity.com/paper/2006.04628/) (2020)
- [DALEX: explainers for complex predictive models](https://mlsanity.com/paper/1806.08915/) (2018)
- [Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models](https://mlsanity.com/paper/1611.04967/) (2016)

## Built on by (most cited first, 2 of 2)

- [Spatial machine-learning model diagnostics: a model-agnostic distance-based approach](https://mlsanity.com/paper/2111.08478/) (2021)
- [Accumulated Aggregated D-Optimal Designs for Estimating Main Effects in Black-Box Models](https://mlsanity.com/paper/2510.08465/) (2025)

## Abstract (arXiv)

Model-agnostic tools for interpreting machine-learning models struggle to summarize the joint effects of strongly dependent features in high-dimensional feature spaces, which play an important role in pattern recognition, for example in remote sensing of landcover. This contribution proposes a novel approach that interprets machine-learning models through the lens of feature space transformations. It can be used to enhance unconditional as well as conditional post-hoc diagnostic tools including partial dependence plots, accumulated local effects plots, or permutation feature importance assessments. While the approach can also be applied to nonlinear transformations, we focus on linear ones, including principal component analysis (PCA) and a partial orthogonalization technique. Structured PCA and diagnostics along paths offer opportunities for representing domain knowledge. The new approach is implemented in the R package `wiml`, which can be combined with existing explainable machine-learning packages. A case study on remote-sensing landcover classification with 46 features is used to demonstrate the potential of the proposed approach for model interpretation by domain experts.