# Predicting into unknown space? Estimating the area of applicability of spatial prediction models

arXiv: https://arxiv.org/abs/2005.07939 · HTML: https://mlsanity.com/paper/2005.07939/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2020.

## Built on by (most cited first, 29 of 29)

- [Better, Not Just More: Data-Centric Machine Learning for Earth Observation](https://mlsanity.com/paper/2312.05327/) (2023)
- [Machine Learning and Deep Learning -- A review for Ecologists](https://mlsanity.com/paper/2204.05023/) (2022)
- [Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction](https://mlsanity.com/paper/2401.06421/) (2024)
- [Comparison of machine learning algorithms for merging gridded satellite and earth-observed precipitation data](https://mlsanity.com/paper/2301.01252/) (2022)
- [Data-Centric Digital Agriculture: A Perspective](https://mlsanity.com/paper/2312.03437/) (2023)
- [mlr3spatiotempcv: Spatiotemporal resampling methods for machine learning in R](https://mlsanity.com/paper/2110.12674/) (2021)
- [Efficient data-driven gap filling of satellite image time series using deep neural networks with partial convolutions](https://mlsanity.com/paper/2208.08781/) (2022)
- [Fine-resolution landscape-scale biomass mapping using a spatiotemporal patchwork of LiDAR coverages](https://mlsanity.com/paper/2205.08530/) (2022)
- [Interactive Generalized Additive Model and Its Applications in Electric Load Forecasting](https://mlsanity.com/paper/2310.15662/) (2023)
- [Earth System Data Cubes: Avenues for advancing Earth system research](https://mlsanity.com/paper/2408.02348/) (2024)
- [Mapping historical forest biomass for stock-change assessments at parcel to landscape scales](https://mlsanity.com/paper/2304.02632/) (2023)
- [The CAST package for training and assessment of spatial prediction models in R](https://mlsanity.com/paper/2404.06978/) (2024)
- [Data-Centric AI for Tropical Agricultural Mapping: Challenges, Strategies and Scalable Solutions](https://mlsanity.com/paper/2510.16207/) (2025)
- [OT on the Map: Quantifying Domain Shifts in Geographic Space](https://mlsanity.com/paper/2604.16220/) (2026)
- [Correcting sampling biases via importance reweighting for spatial modeling](https://mlsanity.com/paper/2309.04824/) (2023)
- [On the use of adversarial validation for quantifying dissimilarity in geospatial machine learning prediction](https://mlsanity.com/paper/2404.12575/) (2024)
- [SatHealth: A Multimodal Public Health Dataset with Satellite-based Environmental Factors](https://mlsanity.com/paper/2506.13842/) (2025)
- [Aligning Validation with Deployment in Spatial Prediction: Target-Weighted Cross-Validation](https://mlsanity.com/paper/2603.29981/) (2026)
- [Conformal PM2.5 Mapping Under Spatial Covariate Shift: Satellite-Reanalysis Fusion for Africa's Green Industrial Transition](https://mlsanity.com/paper/2604.22787/) (2026)
- [FLUXtrapolation: A benchmark on extrapolating ecosystem fluxes](https://mlsanity.com/paper/2605.19812/) (2026)
- [Data-efficient flood depth prediction through domain-aware coreset selection and tabular foundation models](https://mlsanity.com/paper/2606.05265/) (2026)
- [STeMP: Spatio-Temporal Modelling Protocol](https://mlsanity.com/paper/2607.20592/) (2026)
- [Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference](https://mlsanity.com/paper/2608.15282/) (2026)
- [The Cost of a Physics Prior Is Bounded by the Ablation Gap](https://mlsanity.com/paper/2608.21059/) (2026)
- [Semi-Supervised Learning under Spatially Biased Sampling](https://mlsanity.com/paper/2609.07982/) (2026)
- [When Big Data Becomes a Curse: Spatial Heterogeneity and the Limits of Learning from Passive Acoustic Monitoring Data](https://mlsanity.com/paper/2609.26125/) (2026)
- [Geospatial embeddings detect old-growth forests but buffered spatial validation narrows their advantage over Sentinel features](https://mlsanity.com/paper/2609.28194/) (2026)
- [When Should a Satellite Estimate Be Changed? Stress-Testing Neural Corrections for Evapotranspiration](https://mlsanity.com/paper/2609.35314/) (2026)
- [Two-stage Residual Learning for Modelling Strength-Informed Optimisation of Thermal Systems](https://mlsanity.com/paper/2610.19539/) (2026)

## Abstract (arXiv)

Predictive modelling using machine learning has become very popular for spatial mapping of the environment. Models are often applied to make predictions far beyond sampling locations where new geographic locations might considerably differ from the training data in their environmental properties. However, areas in the predictor space without support of training data are problematic. Since the model has no knowledge about these environments, predictions have to be considered uncertain. Estimating the area to which a prediction model can be reliably applied is required. Here, we suggest a methodology that delineates the "area of applicability" (AOA) that we define as the area, for which the cross-validation error of the model applies. We first propose a "dissimilarity index" (DI) that is based on the minimum distance to the training data in the predictor space, with predictors being weighted by their respective importance in the model. The AOA is then derived by applying a threshold based on the DI of the training data where the DI is calculated with respect to the cross-validation strategy used for model training. We test for the ideal threshold by using simulated data and compare the prediction error within the AOA with the cross-validation error of the model. We illustrate the approach using a simulated case study. Our simulation study suggests a threshold on DI to define the AOA at the .95 quantile of the DI in the training data. Using this threshold, the prediction error within the AOA is comparable to the cross-validation RMSE of the model, while the cross-validation error does not apply outside the AOA. This applies to models being trained with randomly distributed training data, as well as when training data are clustered in space and where spatial cross-validation is applied. We suggest to report the AOA alongside predictions, complementary to validation measures.