# Spatial machine-learning model diagnostics: a model-agnostic distance-based approach

arXiv: https://arxiv.org/abs/2111.08478 · HTML: https://mlsanity.com/paper/2111.08478/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 48 references that are in the map.
- First posted on arXiv 13 Nov 2021.

## Builds on (most cited first, 4 of 4)

- [Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance](https://mlsanity.com/paper/1905.03151/) (2019)
- [Estimating the Prediction Performance of Spatial Models via Spatial k-Fold Cross Validation](https://mlsanity.com/paper/2005.14263/) (2020)
- [mlr3spatiotempcv: Spatiotemporal resampling methods for machine learning in R](https://mlsanity.com/paper/2110.12674/) (2021)
- [Transforming Feature Space to Interpret Machine Learning Models](https://mlsanity.com/paper/2104.04295/) (2021)

## Built on by (most cited first, 2 of 2)

- [Evaluation Challenges for Geospatial ML](https://mlsanity.com/paper/2303.18087/) (2023)
- [Aligning Validation with Deployment in Spatial Prediction: Target-Weighted Cross-Validation](https://mlsanity.com/paper/2603.29981/) (2026)

## Abstract (arXiv)

While significant progress has been made towards explaining black-box machine-learning (ML) models, there is still a distinct lack of diagnostic tools that elucidate the spatial behaviour of ML models in terms of predictive skill and variable importance. This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools for spatial prediction models with a focus on prediction distance. Their suitability is demonstrated in two case studies representing a regionalization task in an environmental-science context, and a classification task from remotely-sensed land cover classification. In these case studies, the SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences but also relevant similarities. Limitations of related cross-validation techniques are outlined, and the case is made that modelers should focus their model assessment and interpretation on the intended spatial prediction horizon. The range of autocorrelation, in contrast, is not a suitable criterion for defining spatial cross-validation test sets. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.